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ABSTRACT 

In 1990, the National Assessment of Educational Progress 
(NAEP) included a Trial State Assessment (TSA) ; for the first time in NAEP ' s 
history, voluntary state-by-state assessments were made. In 1992, the NAEP 
Program included an expanded TSA in fourth- grade reading. The 1992 reading 
assessment considered students 1 performance in situations that involved 
reading different kinds of materials for different purposes. The fourth-grade 
assessment measured two global purposes for reading- -reading for literary 
experience and reading to gain information. In Arkansas, 2,589 students in 
116 public schools were assessed. This report describes the reading 
proficiency of Arkansas fourth-graders and compares their overall performance 
to students in the Southeast region of the United States and the nation 
(using data from the NAEP national assessments) . The distribution of reading 
results and reading achievement level results are provided for subpopulations 
of students (race/ethnicity, type of location, parents' educational level, 
and gender, and performance according to purpose for reading) . To provide a 
context for the assessment data, participating public school students, their 
reading teachers, and principals completed questionnaires which focused on: 
policies and practices related to reading (time for instructional activities 
and instructional resources for reading) ,* delivery of reading instruction 
(instructional materials and activities, workbooks/worksheets/writing, 
discussion and group activities, time to read, reading and use of libraries, 
assessing progress in reading) ,* reading instructors (preparation, experience 
and professional development); students' home support for literacy (reading 
outside of school and in the home and hours of television watched per day) . 
The average reading proficiency of fourth-grade public school students in 
Arkansas on the NAEP r f a fttttigr*»ca3re ITnn ^Tpp t ed tp 216 nationwide. (SR) 
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What is The Nation’s Report Card ? 



THE NATION’S REPORT CARD, the National Assessment of Educational Progress (NAEP), is the only nationally representative and continuing assessment 
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selecting the subject areas to be assessed, which may include adding to those specified by Congress; identifying appropriate achievement goals for each age 
and grade; developing assessment objectives; developing test specifications; designing the assessment methodology; developing guidelines and standards 
for data analysis and for reporting and disseminating results; developing standards and procedures for interstate, regional, and national comparisons; improving 
the form and use of the National Assessment; and ensuring that all items selected for use in the National Assessment are free from racial, cultural, gender, 
or regional bias. 
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The National Assessment of Educational Progress (NAEP) is a Congressionally mandated 
project of the National Center for Education Statistics (NCES) that has collected and 
reported information for nearly 25 years on what American students know and what they 
can do. It is the nation’s only ongoing, comparable, and representative assessment of 
student achievement. Its assessments are given to scientifically selected samples of youths 
attending both public and private schools and enrolled in grades four, eight, or twelve. The 
assessment questions are written around a framework prepared for each content area — 
reading, writing, mathematics, science, and others -- that represents the consensus of groups 
of curriculum experts, educators, members of the general public, and user groups on what 
should be covered on such an assessment. Reporting includes means and distributions of 
scores, as well as more descriptive information about the meaning of the data. 



New Reading Assessment Framework and Questions 



The goal of the National Center for Education Statistics is to make data available for the 
public and to do so in accurate and understandable ways that are not misleading. The task 
is challenging because much of what matters in NAEP is changing: 



• the content in response to the developing standards of various curricular 
groups; 

• the assessment questions in response to new developments in assessments; 
and 

• the reporting in response to increasing interest in student achievement 
relative to standards of student performance. 
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The framework for the 1992 Trial State Assessment Program in reading considered 
students’ performance in situations that involved reading different kinds of materials for 
different purposes. The fourth-grade reading assessment measured two global purposes for 
reading — reading for literary experience and reading to gain information. (The eighth- and 
twelfth-grade national NAEP reading assessments also measured a third purpose for 
reading — reading to perform a task.) Reading for literary experience usually involves the 
reading of novels, short stories, plays, and essays. In these reading situations, the reader 
can determine how the author explores or uncovers experiences through the text and 
considers the interplay among events, emotions, and possibilities. Reading to gain 
information usually involves the reading of articles in magazines and newspapers, chapters 
in a textbook, entries in encyclopedias and catalogs, and entire books on particular topics. 
These reading situations call for different orientations to text from those in reading for 
literary experience because readers are specifically focused on acquiring information. 

The assessment asks students to build, extend, and examine text meaning from four stances 
or orientations: 

♦ Initial Understanding -- comprehending the overall or general meaning of 
the selection. 

♦ Developing an Interpretation - extending the ideas in the text by making 
inferences and connections. 

♦ Personal Response - making explicit connections between ideas in the text 
and a student’s own background knowledge and experiences. 

♦ Critical Stance - considering how the author crafted a text. 

These stances are not considered hierarchical or completely independent of each other, but 
are iterative. They provide a frame for generating questions and considering student 
performance at all levels. 

The 1992 NAEP reading assessment uses a variety of innovative assessment approaches 
that are considered significant advancements over previous assessments. In addition to 
multiple-choice questions, the assessment primarily includes constructed-response 
questions that ask students to demonstrate comprehension beyond a surface level. Also, 
longer and naturally-occurring reading materials are used to provide more realistic reading 
experiences than in previous assessments. 

Taken together, the changes in the 1992 reading framework and assessment activities 
preclude any comparisons between the results in this report and those for previous NAEP 
reading assessments. If the current NAEP framework is used in the future, as planned in 
the 1994 assessment, the 1992 reading data will supply the basis for a trend report. 
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A Transition in Reporting 

Over time there have been many changes in emphasis of NAEP reporting, both to take 
advantage of new technologies and to reflect changing trends in education. In 1984, a new 
technology called Item Response Theory (IRT) made it possible to create “scale scores” 
for NAEP similar to those the public was accustomed to seeing for the annual Scholastic 
Aptitude Test (SAT). Educational Testing Service, in its role as Government grantee 
carrying out NAEP operations, devised a new way to describe performance against this 
scale, called “anchor levels.” Starting in 1984, NAEP results were reported by “anchor 
levels.” Anchor levels describe distributions of performance at selected points along the 
NAEP scale (i.e., standard deviation units). Anchor levels show how groups of students 
perform relative to each other, but not whether this performance is adequate. 

In 1988, Congress established the National Assessment Governing Board (NAGB), 
assigning it broad policy making authority over NAEP, including the authority to take 
“appropriate actions ... to improve the form and use of the National Assessment” and to 
identify “appropriate achievement goals for each . . . grade and subject area to be tested in 
the National Assessment.” To carry out its responsibilities, NAGB developed 
"achievement levels,” which are collective judgments about how students should perform 
relative to a body of content reflected in the NAEP frameworks. The result is translated 
onto ranges along the NAEP scale. For the 1992 reading assessment, this process was 
conducted for NAGB under contract by American College Testing (ACT), which has 
extensive experience in standard-setting in many fields. 

With this background, the initial reports for the 1992 reading assessment mark NCES’s 
continued attempt to shift to standards-based reporting of National Assessment statistics. 
The first transition to reporting NAEP results by achievement levels was for the NAEP 
1992 Trial State Assessment in mathematics. 1 The impetus for this transition lies in the 
belief that NAEP data will take on more meaning for the public if they show what 
proportion of our youth are able to meet judgmental standards of performance. 

Reporting NAEP results on the basis of achievement levels represents a significant change 
in practice for NCES. On occasion, this agency makes use of emerging analytical 
approaches that permit new, and sometimes controversial, analyses to be done. When 
doing so, this agency, just as other statistical agencies do when introducing new measures 
to supplement or replace old measures, also has provided the data according to the earlier 
procedures in addition to the new ones. In the case of the 1992 mathematics assessment, 
for example, the “anchor levels” or “scale anchoring” method of reporting was presented 
in an appendix. 



1 For a summary of Ihe 1992 assessment of malhematics, see NAEP 1992 Mathematics Report Card for the 
Nation and the States. (Washinglon, DC: National Center for Education Statistics, 1993) and the individual 
1992 Mathematics State Reports. 
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In this assessment, the “scale anchoring” methodology used by NAEP since 1985 has been 
used but in a new way. As implemented for this report, the scale anchoring process applies 
not to regular scale intervals (standard deviation units), but to the achievement levels 



Appendix D. The critical distinction here is that setting achievement levels attempts to 
describe what students should be able to do in various ranges of the NAEP scale while the 
anchoring procedure attempts to describe what they can do at those achievement levels 
using actual student performance data from the NAEP assessments. 

Chapter 1 of this report describes how the 1992 standards were prepared and provides 
examples of assessment questions that illustrate the reading content reflected in the 
descriptions of the NAEP achievement levels. Chapters 1 - 6 include information on 
overall means, distributions of reading proficiency, as well as background questionnaire 
data, all taken directly from the results of the assessment questions. 

Continuing Development Effort 

We believe that the numerous completed and ongoing studies 3 will lead to national debate 
that can assure the public is well informed about these issues — as informed they must be 
because the results will be a vital influence on what Americans come to think about the 
condition and progress of our schools. Indeed, measures of student learning may be as 
significant bases for public understanding about our nation's education system as the 
Consumer Price Index and the monthly unemployment statistics are in informing the 
public about our nation’s economy. 

In addition, members of the public need the data in this report to see for themselves what 
standards-based reporting might do and to evaluate the often conflicting claims of adherents 
and detractors of these changes in approaches to reporting on the educational achievement 
of American students. Reporting NAEP results to the public would be more clear if the 
language of the achievement levels, or standards, could also directly describe what students 
know and can do. In order to accomplish that, the frameworks, assessment questions, and 
achievement levels may need to be developed in tandem. That is easier to say than to do, 
however, because it implies a substantially larger pool of assessment questions, carefully 
designed to support reporting about performance relative to a set of performance standards. 
Clearly this is a developmental effort that will take time and several iterations, during which 
data supporting appropriate inferences about the performance of American students will 
be gathered on a continuing basis. 



2 First, students were identified who performed at or around the three achievement levels on the scale (212, 243, 
and 275). Next, questions were identified that were answered correctly by 65 percent or more of the 
fourth-grade students at the cutpoint for that achievement level. Finally, reading educators were asked to 
analyze each anchor-level question and create summary descriptions of the skills and abilities evidenced by 
students who answered these sets of questions successfully. 

3 Educational Achievement Standards: Setting Achievement Levels for the Nation. The Second Report of the 
National Academy of Education Panel on the Evaluation of the NAEP Trial State Assessment: 1992 Trial 
State Assessment. (Stanford, CA: National Academy of Education, 1993).; U.S. General Accounting Office 
(GAO). Educational Achievement Standards: NAGB's Approach Yields Misleading Interpretations. June 
1993. GAO/PEMD-93-12.; Assessing Student Achievement in the States. The First Report of the National 
Academy of Education Panel on the Evaluation of the NAEP Trial State Assessment: 1990 Trial State 
Assessment. (Stanford, CA: National Academy of Education, 1992).; R.L. Linn, D.M. Koretz, E.L. Baker, 
and L. Burstein. The Validity and Credibility of the Achievement Levels for the 1990 National Assessment of 
Educational Progress in Mathematics. (Los Angeles, CA: Center for Research on Evaluation, Standards, and 
Student Testing, UCLA, June 1991) CRESST Report 330. 



established for fourth-grade students. 2 The details of this procedure are presented in 
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A Recent History of NAEP 

In 1988, Congress passed new legislation for the National Assessment of Educational 
Progress (NAEP) that continued its primary mission of providing dependable and 
comprehensive information about educational progress in the United States. In addition, 
for the first time in the project’s history, the legislation also included a provision 
authorizing voluntary state-by- state assessments on a trial basis. 

As a result of the legislation, the 1990 NAEP program included a Trial State Assessment 
Program in which public- school students in 37 states, the District of Columbia, and two 
territories were assessed in eighth-grade mathematics. 4 The 1992 NAEP program included 
an expanded Trial State Assessment Program in fourth-grade reading and fourth- and 
eighth-grade mathematics, with public- school students assessed in 41 states, the District of 
Columbia, and two territories. 5 In addition, national assessments in mathematics, reading, 
writing, and science were conducted concurrently with the Trial State Assessment Program 
in 1990 and in 1992. 

School and Student Participation in the Reading Assessment 

In Arkansas, 1 16 public schools participated in the fourth-grade reading assessment. The 
weighted school participation rate was 96 percent, which means that the fourth-grade 
students in this sample of schools were directly representative of 96 percent of all the 
fourth-grade public-school students in Arkansas. 



O 
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4 For a summary of the 1990 program, see Ina V.S. Mullis, John A. Dossey, Eugene H. Owen, and Gary W. 
Phillips. The State of Mathematics Achievement: NAEP’s 1990 Assessment of the Nation and the Trial 
Assessment of the States. (Washington, DC: National Center for Education Statistics, 1991). 



5 For a summary of the 1992 assessment of mathematics, see NAEP 1992 Mathematics Report Card for the 
Nation and the States. (Washington, DC: National Center for Education Statistics, 1993). 
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In total, 2,589 fourth-grade Arkansas public-school students were assessed. The weighted 
student participation rate was 96 percent. This means that the sample of students who 
took part in the assessment was directly representative of 96 percent of the eligible 
fourth-grade public-school student population in participating schools in Arkansas (that is, 
all students from the population represented by the participating schools, minus those 
students excluded from the assessment). 

The overall weighted response rate (school rate times student rate) was 93 percent. This 
means that the sample of students who participated in the assessment was directly 
representative of 93 percent of the eligible fourth-grade public-school student population 
in Arkansas. 



Students’ Reading Performance 



As shown in the following figure, the overall average proficiency of fourth-grade 
public-school students from Arkansas on the NAEP reading scale was 212. This 
proficiency was lower than that of students across the nation (216). 6 There also was a 
tremendous range in student performance. The lowest performing 10 percent of the fourth 
graders from Arkansas had proficiency levels below 164 while the top 10 percent of the 
fourth graders had proficiency levels above 256. 

THE I 
REPORT 

Fourth-Grade Public-School Students’ CARD 

Average Reading Proficiency 

Trial State Assessment 





The NAEP reading scale ranges from 0 to 500. The standard errors are presented in parentheses. With about 
95 percent confidence, the average reading proficiency for each population of interest is within ± 2 standard 
errors of the estimated mean (95 percent confidence interval, denoted by HH). If the confidence intervals for 
the populations do not overlap, there is a statistically significant difference between the populations. If they do 
overlap, the difference may or may not be statistically significant. Statistical tests comparing the two estimates 
must be conducted that use the standard error of the difference (see Appendix A for details). 



6 Differences reported as significant are statistically different at the 95 percent confidence level. This means that 
with 95 percent confidence there is a real difference in the average reading proficiency between the two 
populations of interest. “About the same” means that no statistically significant difference was found at the 
95 percent confidence level. 
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LEVELS OF ACHIEVEMENT 

When Congress established the National Assessment Governing Board (NAGB) in 1988 
to set policy for NAEP, it charged the board with “identifying appropriate achievement 
goals for each age and grade in each subject area to be tested under the National 
Assessment.” (Pub. L. 297-100 Section 3403 (a)(5)(B)(ii)). 

NAGB developed three achievement levels for each grade — Basic, Proficient, and 
Advanced. Performance at the Basic level denotes partial mastery of the knowledge and 
skills that are fundamental for proficient work at each grade level. The central level, called 
Proficient, represents solid academic performance at each grade level tested. Students 
reaching this level demonstrate competency over challenging subject matter and are well 
prepared for the next level of schooling. Achievement at the Advanced level signifies 
superior performance at the grade tested. Definitions of the three levels of reading 
achievement are given below. 



MHB 

level 


Fourth-grade students performing at the Basic level should demonstrate an 
understanding of the overall meaning of what they read. When reading texts? 
appropriate for fourth graders, they should be able to make relatively obvious 
connections between the text and their own experiences. ) 




Fourth-grade students performing at the Proficient level should be able to 
demonstrate an overall understanding of the text , providing inferential as well 
as literal information. When reading text appropriate to fourth grade, they 
should be able to extend the ideas in the text by making inferences, drawing 
conclusions, and making connections to their own experiences. The connection 
between the text and what the student infers should be clear. 




Fourth-grade students performing at the Advanced level should be able to 
generalize about topics in the reading selection and demonstrate an awareness 
of how authors compose and use literary devices. When reading text 
appropriate to fourth grade, they should be able to judge texts critically and, in 
general, give thorough answers that indicate careful thought. 



Because the process of setting the levels of reading achievement centered on the descriptions 
of what students should be able to do, it is important to explore whether students actually 
met the expectations for performance at the Basic, Proficient, and Advanced levels. To 
help in this process, NCES arranged for ETS to apply a modified anchoring procedure to 
the 1992 reading achievement levels. A committee of reading education experts was 
assembled to review the questions and assessment results. Using their knowledge of reading 
and student performance on the individual questions, the committee members were asked 
to summarize student performance at each achievement level (see Appendix D for more 
details on the anchoring procedure). 
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Placing the descriptions of how students performed at each of the levels in the context of 
the expectations for achievement at each of the levels and cross-checking with the actual 
question-by-question results yields some interesting findings. In general, the sets of reading 
skills expected were those observed. However, in some instances, particularly for extended 
response questions, even Advanced-level students had difficulty providing in-depth answers. 
In some other instances, because the assessment was developed prior to the achievement 
level descriptions, particular reading skills were not measured. For more information about 
student performance, see the full report. 
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The following figure provides the percentage of students at or above each achievement 
level, as well as the percentage of students below the Basic level. In Arkansas, 53 percent 
of the fourth graders in public schools were at or above the Basic level, 20 percent were 
at or above the Proficient level, and 3 percent were at or above the Advanced level. 
Nationwide, 57 percent of the fourth graders were at or above the Basic level, 24 percent 
were at or above the Proficient level, and 4 percent were at or above the Advanced level. 
A somewhat smaller percentage of students in Arkansas than across the nation were at or 
above the Proficient level. 



Levels of Fourth-Grade Public-School 
Students’ Reading Achievement 



THE NATION’S 
REPORT 



CARD 



rasp 

m 



1992 

Trial State Assessment 



State 

Region 

Nation 



State 

Region 

Nation 



State 

Region 

Nation 



State 

Region 

Nation 




Percent 

3 ( 0 . 4 ) 

3 ( 0 . 6 ) 

4 ( 0 . 6 ) 



20 ( 1 . 3 ) 
19 ( 2 . 4 ) 
24 ( 1 . 2 ) 



53 ( 1 . 6 ) 
52 ( 3.5) 
57 ( 1 . 2 ) 



47 ( 1 . 6 ) 

48 ( 3.5) 
43 ( 1 . 2 ) 



PERCENT 

The standard errors are presented in parentheses. With about 95 percent confidence, the 
average reading proficiency for each population of interest is within ± 2 standard errors of 
the estimated mean (95 percent confidence interval, denoted by HH). If the confidence 
intervals for the populations do not overlap, there is a statistically significant difference 
between the populations. If they do overlap, the difference may or may not be statistically 
significant. Statistical tests comparing the two estimates must be conducted that use the 
standard error of the difference (see Appendix A for details). 
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PERFORMANCE ACCORDING TO PURPOSE FOR READING 

The 1992 Trial State Assessment Program considered students’ performance in situations 
that involved reading different kinds of materials for different purposes. The fourth-grade 
reading assessment measured two global purposes for reading — reading for literary 
experience and reading to gain information. Students in Arkansas performed lower than 
students across the nation in reading for literary experience. 



Subpopulation Performance 



Assessment results repeatedly show differences in achievement for subpopulations of 
students. 7 The 1992 Trial State Assessment provides additional information about the 
achievement of important subpopulations by reporting on the performance of various 
subgroups of the student population defined by race/ethnicity, type of community, parents’ 
education level, and gender. 



Race/Ethnicity 



White students in Arkansas demonstrated higher average reading proficiency than 
did Black, Hispanic, or American Indian students. In Arkansas, about one quarter 
of the White students (25 percent), relatively few of the Black students 
(5 percent), relatively few of the Hispanic students (6 percent), and some of the 
American Indian students (14 percent) were at or above the Proficient level. 
Across the nation, about one quarter of the White students (30 percent), relatively 
few of the Black students (7 percent), some of the Hispanic students (12 percent), 
and some of the American Indian students (13 percent) were at or above the 
Proficient level. 



Type of Community 



The average reading performance of Arkansas students attending schools in areas 
classified as “other” was higher than that of students attending schools in 
disadvantaged urban areas and about the same as that of students attending 
schools in extreme rural areas. Some of the students attending schools in areas 
classified as “other” (20 percent), some of the students in disadvantaged urban 
areas (12 percent), and some of the students in extreme rural areas (18 percent) 
in Arkansas were at or above the Proficient level. Across the nation, about one 
quarter of the students in areas classified as “other” (24 percent), relatively few of 
the students in disadvantaged urban areas (5 percent), and about one quarter of 
the students in extreme rural areas (24 percent) were at or above the Proficient 
level. 



7 Ina V.S. Mullis, John A. Dossey, Mary A. Foertsch, Lee R. Jones, and Claudia A. Gentile. Trends in 
Academic Progress. (Washington, DC: National Center for Education Statistics, 1992). 
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Parents' Education 



Students in Arkansas who reported that at least one parent graduated from college 
demonstrated about the same average reading proficiency as did students who 
reported that at least one parent had some education after high school or at least 
one parent graduated from high school, but higher proficiency than did students 
who reported that neither parent graduated from high school or they did not know 
their parents’ education level. Reading achievement in Arkansas was at or above 
the Proficient level for 26 percent of the students who reported that at least one 
parent graduated from college, 30 percent of the students who reported that at 
least one parent had some education after high school, 19 percent of the students 
who reported that at least one parent graduated from high school, 12 percent of 
the students who reported that neither parent graduated from high school, and 
12 percent of the students who reported that they did not know their parents’ 
education level. Across the nation, these figures were 33 percent of the students 
who reported that at least one parent graduated from college, 28 percent of the 
students who reported that at least one parent had some education after high 
school, 18 percent of the students who reported that at least one parent graduated 
from high school, 10 percent of the students who reported that neither parent 
graduated from high school, and 17 percent of the students who reported that they 
did not know their parents’ education level. 



Gender I I n Arkansas, fourth-grade boys attending public schools had a lower average 
reading proficiency than did fourth-grade girls. Compared to the national results, 
girls in Arkansas performed lower than girls across the country; boys in Arkansas 
performed about the same as boys across the country. There was a significant 
difference between the percentages of males and females in Arkansas who attained 
the Proficient level (22 percent for females and 17 percent for males). The 
percentage of females in Arkansas who attained the Proficient level was about the 
same as the percentage of females in the nation who attained the Proficient level 
(22 percent for Arkansas and 26 percent for the nation). However, the percentage 
of males in Arkansas who attained the Proficient level was somewhat lower than 
the percentage of males in the nation who attained the Proficient level (17 percent 
for Arkansas and 21 percent for the nation). 



A Context for Understanding Students’ Reading Proficiency 

Information on the reading performance of students in Arkansas can be better understood 
and used for improving instruction and setting policy when supplemented with contextual 
information about schools, teachers, and students. 

To gather contextual information, the fourth-grade students participating in the 1992 Trial 
State Assessment, their reading teachers, and the principals or other administrators in their 
schools were asked to complete questionnaires on policies, instruction, and programs. The 
student, teacher, and school data help to describe some of the current practices and 
emphases in reading education, illuminate some of the factors that appear to be related to 
fourth-grade public-school students’ reading proficiency, and provide an educational 
context for understanding information on student achievement. Highlights of the results 
for the public-school students in Arkansas are as follows: 
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CURRICULUM COVERAGE AND INSTRUCTIONAL EMPHASIS 

• In Arkansas, 55 percent of the fourth-grade students had reading teachers 
who spent at least 60 minutes providing reading instruction each day. By 
comparison, 45 percent of the students had reading teachers who spent 
45 minutes or less providing reading instruction each day. 

• In Arkansas, some of the fourth-grade students (15 percent) were being 
taught by teachers who placed heavy emphasis on phonics; less than half 
(3 1 percent) were being taught by teachers who placed heavy emphasis on 
the integration of reading and writing; and about one quarter (21 percent) 
were being taught by teachers who placed heavy emphasis on the whole 
language approach. 

• In addition, in Arkansas, about one quarter of the fourth-grade students 
(24 percent) were being taught by teachers who placed heavy emphasis on 
literature- based reading; less than half (39 percent) were being taught by 
teachers who placed heavy emphasis on reading across the content areas; 
and some (12 percent) were being taught by teachers who placed heavy 
emphasis on individualized reading programs. 



DELIVERY OF READING INSTRUCTION 

• Less than half of the fourth-grade public-school students in Arkansas 
(35 percent) had reading teachers who used both basal and trade books, 
more than half (63 percent) had reading teachers who primarily used basal 
readers, and none (0 percent) had reading teachers who primarily used 
trade books. 

• In Arkansas, 2 percent of the fourth-grade students had reading teachers 
who used children’s newspapers and/or magazines almost every day; 
1 1 percent of the students had reading teachers who used reading kits 
almost every day; 4 percent had reading teachers who used computer 
software for reading instruction almost every day; 23 percent had reading 
teachers who used a variety of books almost every day; and, finally, 
14 percent had teachers who used materials from other subject areas 
almost every day. 

• In Arkansas, about one quarter of the fourth-grade students (24 percent) 
had reading teachers who devoted almost all of their instructional time in 
reading to teaching decoding skills; about one quarter of the students 
(25 percent) had reading teachers who devoted almost all of their 
instructional time in reading to oral reading; less than half (41 percent) had 
reading teachers who devoted almost all of their instructional time in 
reading to teaching vocabulary; more than half (57 percent) had reading 
teachers who devoted almost all of their instructional time in reading to 
comprehension/interpretation; and finally, less than half (31 percent) had 
teachers who devoted almost all of their instructional time in reading to 
reading strategies. 
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EDUCATIONAL BACKGROUND OF TEACHERS 

• In Arkansas, 33 percent of the students were being taught by reading 
teachers who reported having at least a master’s or education specialist’s 
degree. This compares with 46 percent for students across the nation. 

• About half of the students (52 percent) had reading teachers who had the 
highest level of teaching certification that is recognized by Arkansas. This 
is about the same as the figure for the nation, where more than half of the 
students (57 percent) were taught by reading teachers who were certified 
at the highest level available in their states. 

• In Arkansas, 14 percent of the fourth-grade public-school students were 
being taught reading by teachers who had an undergraduate major in 
English, reading, and/or language arts. By comparison, 22 percent of the 
students across the nation had reading teachers with the same major. 

HOME FACTORS 

• Students in Arkansas who had four types of reading materials in the home 
(newspapers, magazines, more than 25 books, and an encyclopedia) 
showed a higher reading proficiency than did students with zero to two 
types of materials. Across the nation, students who had all four types of 
materials at home showed a higher reading proficiency than did students 
who had zero to two types. 

• In Arkansas, 31 percent of the fourth-grade public-school students 
discussed what they read with friends or family almost every day; 

24 percent never or hardly ever discussed what they read. Across the 
nation, 27 percent discussed what they read with friends or family almost 
every day and 24 percent never or hardly ever discussed what they read. 

• Some of the fourth-grade public-school students in Arkansas (16 percent) 
watched one hour or less of television each day; about one quarter 
(28 percent) watched six hours or more. 

Comparisons of Overall Reading Proficiency in Arkansas with Other 
States 

The map on the following page provides a method for making appropriate comparisons 
of the overall reading proficiency in Arkansas with that in other states (including the 
District of Columbia and one territory) that participated in the NAEP 1992 Trial State 
Assessment Program. The different shadings of the states on the map show whether the 
average overall proficiency in the other states was statistically different from or not 
statistically different from that in Arkansas (“Target State”). States with a dark-colored 
shading have a significantly higher average proficiency than does Arkansas. States with a 
light-colored shading have a significantly lower average proficiency than does Arkansas. 
States without shading have an average proficiency that does not differ significantly from 
that of Arkansas. The significance tests are based on a Bonferroni procedure for multiple 
comparisons that holds the probability of erroneously declaring the means of any two states 
to be different, when they are not, to no more than five percent. 
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OVERVIEW 




In 1988, Congress passed new legislation for the National Assessment of Educational 
Progress (NAEP) that continued its primary mission of providing dependable and 
comprehensive information about educational progress in the United States. In addition, 
for the first time in the project’s history, the legislation also included a provision 
authorizing voluntary state-by-state assessments on a trial basis: 

The National Assessment shall develop a trial mathematics assessment survey 
instrument for the eighth grade and shall conduct a demonstration of the 
instrument in 1990 in States which wish to participate, with the purpose of 
determining whether such an assessment yields valid, reliable State representative 
data . (Section 406(i)(2)(C)(i) of the General Education Provisions Act, as 
amended by Pub . L. 100-297 (U.S.C. 1221e-l (i)(2)(c)(i))) 

The National Assessment shall conduct a trial mathematics assessment for the 
fourth and eighth grades in 1992 and, pursuant to subparagraph (6)(D), shall 
develop a trial reading assessment to be administered in 1992 for the fourth grade 
in States which wish to participate, with the purpose of determining whether such 
an assessment yields valid, reliable State representative data. (Section 
406(i)(2)(C)(i) of the General Education Provisions Act, as amended by Pub. 

L. 100-297 (U.S.C. 1221e-l(i)(2)(c)(ii))) 



As a result of the legislation, the 1990 NAEP program included a Trial State Assessment 
Program in which public-school students in 37 states, the District of Columbia, and two 
territories were assessed in eighth-grade mathematics. 8 The 1992 NAEP program included 
an expanded Trial State Assessment Program in fourth-grade reading and fourth- and 
eighth-grade mathematics, with public-school students assessed in 41 states, the District of 
Columbia, and two territories. 9 In addition, national assessments in mathematics, reading, 
writing, and science were conducted concurrently with the Trial State Assessment Program 
in 1990 and in 1992. 



8 For a summary of the 1990 program, see Ina V.S. Mullis, John A. Dossey, Eugene H. Owen, and Gary W. 
Phillips. The State of Mathematics Achievement: NAEP's 1990 Assessment of the Nation and the Trial 
Assessment of the States. (Washington, DC: National Center for Education Statistics, 1991). 

9 For a summary of the 1992 assessment of mathematics, see NAEP 1992 Mathematics Report Card for the 
Nation and the States. (Washington, DC: National Center for Education Statistics, 1993). 
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The 1992 Trial State Assessment Program was conducted in February 1992 with the 
following 44 participants: 
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* The Virgin Islands participated in the testing portion of the 1992 Trial State Assessment Program. However, 
in accordance with the legislation providing for participants to review and give permission for release of their 
results, the Virgin Islands chose not to release their results at grade 4 in the reports. 



States in regular type did not participate in the 1990 Trial State Assessment. Three states 
— Montana, Illinois, and Oregon — participated in the 1990 Trial State Assessment but not 
in the 1992 program. 

For the 1992 Trial State Assessment in reading, approximately 2,500 students were assessed 
in each jurisdiction. The samples were carefully designed to represent the fourth-grade 
public-school populations in the states or territories. Similar to the 1990 program, local 
school district personnel administered all assessment sessions, and the contractor’s staff 
monitored 50 percent of the sessions as part of the quality assurance program designed to 
ensure that the sessions were conducted uniformly. The results of the monitoring in 1990 
and 1992 indicated a high degree of quality and uniformity across sessions. 

The 1992 Trial State and National Assessment programs in reading were based on a 
framework developed through a national consensus process that was set forth by law and 
called for “active participation of teachers, curriculum specialists, subject matter specialists, 
local school administrators, parents, and members of the general public” (Public Law 
100-297, Part C, 1988). 10 



10 Reading Framework for the 1992 National Assessment of Educational Progress. (Washington, DC: National 
Assessment Governing Board, U.S. Department of Education, 1992). 
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The process of developing the framework was carried out in late 1989 and early 1990 by 
the Council of Chief State School Officers (CCSSO) under contract from the National 
Assessment Governing Board (NAGB) which is responsible for formulating policy for 
NAEP, including developing assessment objectives and test specifications. The framework 
development process included input from a wide range of people in the fields of reading 
and assessment, from school teachers and administrators to state coordinators of reading 
and reading assessment. After thorough discussion and some amendment, the framework 
was adopted by NAGB in March 1990. An overview of the reading framework is provided 
in the Procedural Appendix. 

The fourth-grade Trial State and National Assessments in reading included eight sections 
or blocks, each 25 minutes in length. Each block consisted of a passage and a combination 
of constructed-response and multiple-choice questions. Passages selected for the 
assessment were drawn from authentic texts used by students in real, everyday reading. 
Whole stories, articles, or sections of textbooks were used, rather than excerpts or 
abridgements. The type of question — constructed-response or multiple-choice -- was 
determined by the nature of the task. In addition, the constructed-response questions were 
of two types: regular constructed-response questions required students to respond to a 
question in a few words or a few sentences while extended constructed-response questions 
required students to respond to a question in a paragraph or more. 



This Report 



This is a computer-generated report that describes the reading performance of fourth-grade 
public-school students in Arkansas, in the Southeast region, and across the nation. A 
separate report describes additional fourth-grade reading assessment results for the nation 
and the states, as well as the national results for grades 8 and 12. 11 This report consists 
of three sections: 



• This Overview provides background information about the Trial State 
Assessment and a profile of the fourth-grade public-school students in 
Arkansas. 

• Part One describes the reading performance of the fourth-grade 
public- school students in Arkansas, the Southeast region, and the nation. 

• Part Two relates fourth-grade students’ reading performance to contextual 
information about the reading policies, instruction, and home support for 
reading in Arkansas, the Southeast region, and the nation. 



11 See NAEP 1992 Reading Report Card for the Nation and the States. (Washington, DC: National Center for 
Education Statistics, 1993). 
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In this report, results are provided for groups of students defined by shared characteristics 
— race/ethnicity, type of community, parents’ education level, and gender. Definitions of 
the subpopulations referred to in this report are presented below. The results for Arkansas 
are based on the representative sample of public-school students who participated in the 
1992 Trial State Assessment Program. The results for the nation and the region of the 
country are based on the nationally and regionally representative samples of public-school 
students who were assessed in January through March as part of the national NAEP 
program. Using the regional and national results from the 1992 national NAEP program 
is necessary because of the voluntary nature of the Trial State Assessment Program. Since 
not every state participated in the program, the aggregated data across states did not 
necessarily provide representative national or regional results. Specific details on the 
samples and analysis procedures used can be found in the Technical Report of the 1992 
NAEP Trial State Assessment Program in Reading} 2 



RACE/ETHNICITY 

Results are presented for students of different racial/ethnic groups based on the students’ 
self-identification of their race/ethnicity according to the following mutually exclusive 
categories: White, Black, Hispanic, Asian (including Pacific Islander), and American 
Indian (including Alaskan Native). Based on criteria described in the Procedural Appendix, 
there must be at least 62 students in a particular subpopulation in order for the results for 
that subpopulation to be considered reliable. Thus, results for racial/ethnic groups with 
fewer than 62 students are not reported. However, the data for all students, regardless of 
whether their racial/ethnic group was reported separately, were included in computing 
overall results for Arkansas. 



TYPE OF COMMUNITY 

Results are provided for four mutually exclusive community types — advantaged urban, 
disadvantaged urban, extreme rural, and other — as defined below: 

Advantaged Urban : Students in this group live in metropolitan statistical areas 
and attend schools where, according to their schools, a high proportion of the 
students’ parents are in professional or managerial positions. 

Disadvantaged Urban: Students in this group live in metropolitan statistical 
areas and attend schools where, according to their schools, a high proportion 
of the students’ parents are on welfare or are not regularly employed. 



12 Technical Report of the NAEP 1992 Trial State Assessment Program in Reading. (Washington, DC: 
National Center for Education Statistics, 1993). 
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Extreme Rural: Students in this group live outside metropolitan statistical 

areas, live in areas with a population below 10,000, and attend schools where, 
according to their schools, many of the students’ parents are farmers or farm 
workers. 

Other: Students in this category attend schools in areas other than those defined 
as advantaged urban, disadvantaged urban, or extreme rural. 

Indices were developed such that approximately 10 percent of the most extreme advantaged 
urban, disadvantaged urban, and rural schools sampled in the national assessment were 
classified into these three categories. The remaining 70 percent of the schools were 
classified into the “other” category. The reporting of results by each type of community 
was also subject to a minimum student sample size of 62. 

PARENTS’ EDUCATION LEVEL 

Students were asked to indicate the extent of schooling for each of their parents — did not 
finish high school, graduated from high school, some education after high school, or 
graduated from college. The response indicating the higher level of education was selected 
for reporting. Reporting of results by parents’ education level was also subject to a 
minimum student sample size of 62. Note that a substantial percentage of fourth-grade 
students did not know their parents’ education level. 

GENDER 

Results are reported separately for males and females. 

REGION 

The United States has been divided into four regions for purposes of this report: Northeast, 
Southeast, Central, and West. States included in each region are shown in Figure 1. All 
50 states and the District of Columbia are listed, with the participants in the Trial State 
Assessment highlighted in boldface type. Territories were not assigned to a region. 
Further, the part of Virginia that is included in the Washington, DC, metropolitan 
statistical area is included in the Northeast region; the remainder of the state is included in 
the Southeast region. Because most of the Virginia students are in the Southeast region, 
regional comparisons for Virginia are to the Southeast. 

The regional results are based on a separate sample from that used to report the state 
results. Regional results are based on national assessment samples, not on aggregated Trial 
State Assessment samples. 




THE 1992 NAEP TRIAL STATE ASSESSMENT 



19 



Arkansas 



FIGURE l I Regions of the Country 
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Guidelines for Analysis 

This report describes reading proficiency for fourth graders attending public schools and 
compares the results for various groups of students within that population — for example, 
those who have certain demographic characteristics or who responded to a specific 
background question in a particular way. The report examines the results for individual 
groups and individual background questions. It does not include an analysis of the 
relationships among combinations of these subpopulations or background questions. 

Because the proportions of students in these subpopulations and their average proficiencies 
are based on samples -- rather than the entire population of fourth graders in public schools 
in a state or territory -- the numbers reported are necessarily estimates . As such, they are 
subject to a measure of uncertainty, reflected in the standard error of the estimate. When 
the proportions or average proficiencies of certain groups are compared, it is essential to 
take the standard error into account, rather than rely solely on observed similarities or 
differences. Therefore, the comparisons discussed in this report are based on statistical tests 
that consider both the magnitude of the difference between the means or proportions and 
the standard errors of those statistics. 
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The statistical tests determine whether the evidence — based on the data from the groups 
in the sample — is strong enough to conclude that the means or proportions are really 
different for those groups in the population. If the evidence is strong (i.e., the difference is 
statistically significant ), the report describes the group means or proportions as being 
different (e.g., one group performed higher than or lower than another group) — regardless 
of whether the sample means or sample proportions appear to be about the same or not. 
If the evidence is not sufficiently strong (i.e., the difference is not statistically significant), 
the means or proportions are described as being about the same ~ again, regardless of 
whether the sample means or sample proportions appear to be about the same or widely 
discrepant. The reader is cautioned to rely on the results of the statistical tests -- rather than 
on the apparent magnitude of the difference between sample means or proportions — to 
determine whether those sample differences are likely to represent actual differences 
between the groups in the population. The statistical tests and Bonferroni procedure, 
which is used when more than two groups are being compared, are discussed in greater 
detail in the Procedural Appendix. 

In addition, some of the percentages reported in the text of the report are given quantitative 
descriptions (e.g., some, about half, almost all, etc.). The descriptive phrases used and the 
rules used to select them are described in the Procedural Appendix. 

Finally, in several places in this report, results (mean proficiencies and proportions) are 
reported in the text for combined groups of students. For example, in the text, the 
percentage of students in the combined group where teachers reported spending 60 minutes 
or 90 minutes or more on reading instruction on a typical day is given and compared to the 
group where teachers reported spending 45 minutes or less. However, the table that 
accompanies that text reports percentages and proficiencies separately for the three groups 
(45 minutes or less, 60 minutes, and 90 minutes or more). The combined group 
percentages reported in the text and used in all statistical tests are based on unrounded 
estimates (i.e., estimates calculated to several decimal places) of the percentages in each 
group. The percentages shown in the tables are rounded to integers. Thus, percentages 
may not always add up to 100 percent due to rounding. Also, the percentage for a 
combined group (reported in the text) may differ slightly from the sum of the separate 
percentages (presented in the tables) for each of the groups that were combined. Therefore, 
if statistical tests were to be conducted based on the rounded numbers in the tables, the 
results might not be consonant with the results of the statistical tests that are reported in 
the text (based on unrounded numbers.) 

Profile of Arkansas 

FOURTH-GRADE SCHOOL AND STUDENT CHARACTERISTICS 

Table 1 provides a profile of the demographic characteristics of the fourth-grade 
public- school students in Arkansas, the Southeast region, and the nation. This profile is 
based on data collected from the students and schools participating in the 1992 Trial State 
and National Assessments. As described earlier, the state data and the regional and 
national data are drawn from separate samples. 
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Profile of Fourth-Grade 
Public-School Students in Arkansas, 
the Southeast Region, and the 
Nation 



Arkansas 



Southeast 



Nation 



DEMOGRAPHIC SUBGROUPS 



Race/Ethnicity 

White 

Black 

Hispanic 

Asian 

American Indian 

Type of Community 

Advantaged urban 
Disadvantaged urban 
Extreme rural 
Other 

Parents’ Education 

Graduated college 
Some education after high school 
Graduated high school 
Did not finish high school 
I don't know 

Gender 

Male 

Female 



* Percentage 

VO { 1.8) 

•’ 21 (1.5) 

7 ( 0.7) 

. ,1 ( 0 . 2 ) , 

||||||||l|2 ( 0.3) : 



ms fm t t.2j 

6 ( 1.5) 

. - 25 ( 4.0) .. 

68 ( 4.2) 

: 32 (1,3) : 

.10(0.7) 
20(0.9) 

' 9 ( , 0 . 8 ) 

30 I 1.0) 

50 ( 1.0) 

' 50 ( 1.0) 



5 : V • ^ ^ 

Percentage 

63(2.7) 

29 ( 2.6) I II 
5 ( 1.1) 
1(0.3) 
1(0.4) 

5 { 3*3) 

19(6.9) 
62(7.5) 

P': » 

36(2*2) . 
8(0.9) 

I | .5 ((0.7) | I 



V 49(1.3) (If 
. 51 ( 1.3) 



iiigisaBtessi j m m . 

ipitfWiftiaiii 

■llilll ( 0.4) ; ; : 

III1110:TOi3piil! 

2 < 0.3) 
7(2-1) 

70 ( 3.2) S : 




4(0.4) 
37 { 1.1) 

: 51 ( 0.7) 

* y % y40 



The standard errors of the statistics appear in parentheses. It can be said with about 95 percent confidence that, 
for each population of interest, the value for the entire population is within ± 2 standard errors of the estimate 
for the sample. In comparing two estimates, one must use the standard error of the difference (see 
Appendix A for details). The percentages for Race/Ethnicity may not add to 100 percent because some 
students categorized themselves as “Other.” 



SCHOOLS AND STUDENTS ASSESSED 

Table 2 summarizes participation data for Arkansas schools and students sampled for the 
1992 Trial State Assessment. 13 In Arkansas, 1 16 public schools participated in the 
fourth-grade reading assessment. These numbers include participating substitute schools 
that were selected to replace some of the nonparticipating schools from the original sample. 
The weighted school participation rate was 96 percent, which means that the fourth- grade 
students in this sample of schools were directly representative of 96 percent of all the 
fourth-grade public-school students in Arkansas. 



13 For a detailed discussion of the NCES guidelines for sample participation, see School and Student 
Participation Rates for the Reading Assessment. (Washington, DC: National Centerfor Education Statistics, 
1993); or see Appendix B of the Technical Report of the NAEP 1992 Trial State Assessment Program in 
Reading. (Washington, DC: National Center for Education Statistics, 1993). 
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In each school, a random sample of students was selected to participate in the assessment. 
As estimated by the sample, 0 percent of the fourth-grade public-school population was 
classified as Limited English Proficient (LEP), while 11 percent in fourth grade had an 
Individualized Education Plan (IEP). An IEP is a plan, written for a student who has been 
determined to be eligible for special education, that typically sets forth goals and objectives 
for the student and describes a program of activities and/or related services necessary to 
achieve the goals and objectives. Handicapped or disabled students may be categorized 
as IEP. 

Schools were permitted to exclude certain students from the assessment, provided that 
certain criteria were met. To be excluded, a student had to be categorized as Limited 
English Proficient or had to have an Individualized Education Plan and (in either case) be 
judged incapable of participating in the assessment. The intent was to assess all selected 
students; therefore, all selected students who were capable of participating in the assessment 
should have been assessed. However, schools were allowed to exclude those students who, 
in the judgment of school staff, could not meaningfully participate. The NAEP guidelines 
for exclusion are intended to assure uniformity of exclusion criteria from school to school. 
Note that some LEP and IEP students were deemed eligible to participate and not excluded 
from the assessment. The students in Arkansas who were excluded from the assessment 
because they were categorized as LEP or had an IEP represented 5 percent of the 
population in grade four. 

In total, 2,589 fourth-grade Arkansas public-school students were assessed. The weighted 
student participation rate was 96 percent. This means that the sample of students who 
took part in the assessment was directly representative of 96 percent of the eligible 
fourth-grade public-school student population in participating schools in Arkansas (that is, 
all students from the population represented by the participating schools, minus those 
students excluded from the assessment). 

The overall weighted response rate (school rate times student rate) was 93 percent. This 
means that the sample of students who participated in the assessment was directly 
representative of 93 percent of the eligible fourth-grade public-school population in 
Arkansas. 

In the analysis of student data and reporting of results, nonresponse weighting adjustments 
have been made at both the school and student level, with the aim of making the sample 
of participating students as representative as possible of the entire eligible fourth-grade 
public-school population. For details of the nonresponse weighting adjustment 
procedures, see the Technical Report of the NAEP 1992 Trial State Assessment Program in 
Reading. 
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TABLE 2 



Profile of the Fourth-Grade 
Population Assessed in Arkansas 



PUBLIC SCHOOL PARTICIPATION 



Weighted school participation rate before substitution 

Weighted school participation rate after substitution 

Number of schools originally sampled 

Number of schools not eligible 

Number of schools in original sample participating 

Number of substitute schools provided 

Number of substitute schools participating 

Total number of participating schools 



PUBLIC-SCHOOL STUDENT PARTICIPATION 



Weighted student participation rate after makeups 

Number of students selected to participate in the assessment 

Number of students withdrawn from the assessment 

Percentage of students who were of Limited English Proficiency 

Percentage of students excluded from the assessment due to Limited 
English Proficiency 

Percentage of students who had an Individualized Education Plan 

Percentage of students excluded from the assessment due to Individualized 
Education Plan Status 

Number of students to be assessed 

Number of students assessed 
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87% 
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; 0 % 

11 % 
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Overall weighted response rate 



In one or more schools in Arkansas an assessment was conducted, but either the wrong materials were sent to 
the school(s) or the materials were lost in shipping via the U.S. Postal Service. The school(s) are included in the 
counts of participating schools in Arkansas, both before and after substitution. However, in the weighted 
results, the school(s) are treated in the same manner as a non-participating school because no student responses 
were available for analysis and reporting. The students in these school(s) are included in the counts of students 
in the table. However, the estimates for Arkansas will not be based on these student responses. 
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How Proficient in Reading are Fourth-Grade 
Students in Arkansas Public Schools? 



Reading involves the interaction between a reader, a text, and a situation. 14 Thus, 
students’ reading comprehension is influenced by the type of material read and the specific 
purposes for reading. The 1992 Trial State Assessment Program considered students’ 
performance in situations that involved reading different kinds of materials for different 
purposes. The fourth-grade reading assessment measured two global purposes for reading 
- reading for literary experience and reading to gain information . 15 Reading for literary 
experience usually involves the reading of novels, short stories, plays, and essays. In these 
reading situations, the reader can determine how the author explores experiences through 
the text and can consider the interplay among events, emotions, and possibilities. Reading 
to gain information usually involves the reading of articles in magazines and newspapers, 
chapters in a textbook, entries in encyclopedias and catalogs, and entire books on particular 
topics. These reading situations call for different orientations to text from those in reading 
for literary experience because readers are specifically focused on acquiring information. 
Students’ performance on each of the two purposes for reading was summarized on 
separate NAEP reading scales (one for each purpose), which range from 0 to 500. In 
addition, an overall reading scale, reflecting combined performance in the two purposes for 
reading, was also developed. The overall reading scale also ranges from 0 to 500. 



14 Judith A. Langer, Arthur N. Applebee, Ina V.S. Mullis, and Mary A. Foertsch. Learning to Read in Our 
Nation’s Schools. (Princeton, NJ: National Assessment of Educational Progress, Educational Testing 
Service, 1990). 

15 The eighth- and twelfth-grade national NAEP reading assessments also measured a third purpose for reading 
- reading to perform a task. 
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This part of the report contains two chapters that describe the reading proficiency of 
fourth-grade public-school students in Arkansas. Chapter 1 compares the overall reading 
performance of the students in Arkansas to students in the Southeast region and the nation. 
It also presents the students’ average proficiency for the two purposes for reading. 

Chapter 2 summarizes the students’ overall reading performance, as well as performance 
for each of the two reading purposes, for subpopulations defined by race/ethnicity, type of 
community, parents’ education level, and gender. 
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CHAPTER 1 

Students’ Reading Performance 

Reading achievement is central to one of the goals adopted by the president and the 
governors following the historic Charlottesville conference — American students will leave 
grades four , eight , and twelve having demonstrated competency in challenging subject matter 
including English , mathematics , science , history, and geography; and every school in America 
will ensure that all students learn to use their minds well, so they may be prepared for 
responsible citizenship, further learning, and productive employment in our modern 
economy} 6 Concern about attaining the goal and, more importantly, about the reading 
abilities of our nation’s students has increased recently because it appears that many 
students of all ages have difficulty reading thoughtfully. 17 

Reading for meaning involves a dynamic, complex interaction between and among the 
reader, the text, and the context. Readers, for example, bring to the process their prior 
knowledge about the topic, their reasons for reading it, their individual reading skills and 
strategies, and their understanding of differences in text structures. 

The texts used in the reading assessment are representative of common reading demands. 
Students in grade 4 are asked to respond to literary and informational texts which differ in 
structure, organization, and features. Literary texts include short stories, poems, and plays 
that engage the reader in a variety of ways, not the least of which is reading for fun. 
Informational texts include selections from textbooks, magazines, encyclopedias, and other 
written sources whose purpose is to increase the reader’s knowledge. 



16 AMERICA 2000: An Education Strategy (Washington, DC: U.S. Department of Education, 1991). 

17 Ina V.S. Mullis, John A. Dossey, Mary A. Foertsch, Lee R. Jones, and Claudia A. Gentile. Trends in 
Academic Progress . (Washington, DC: National Center for Education Statistics, 1992); Judith A. Langer, 
Arthur N. Applebee, Ina V.S. Mullis, and Mary A. Foertsch. Learning to Read in Our Nation's Schools. 
(Princeton, NJ: National Assessment of Educational Progress, Educational Testing Service, 1990); Richard 
C. Anderson, Elfrieda H. Hiebert, Judith A. Scott, Ian A. Wilkinson. Becoming a Nation of Readers. 
(Washington, DC: National Institute of Education, 1985). 
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The context of the reading situation includes the purposes for reading that the reader might 
use in building a meaning of the text. For example, in reading for literary experience, 
students may want to see how the author explores or uncovers experiences, or they may 
be looking for vicarious experience through the story’s characters. On the other hand, the 
student’s purpose in reading informational texts may be to learn about a topic (such as the 
Civil War or the oceans) or to search for specific information. 



The assessment asks students to build, extend, and examine text meaning from four stances 
or orientations: 



Initial 

Understanding 



Students are asked to provide the overall or general meaning of the selection. This 
includes summaries, main points, or themes. 



Developing an 
Interpretation 



Students are asked to extend the ideas in the text by making inferences and 
connections. This includes making connections between cause and effect, 
analyzing the motives of characters, and drawing conclusions. 



Personal 

Response 



Students are asked to make explicit connections between the ideas in the text and 
their own background knowledge and experiences. This includes comparing story 
characters with themselves or people they know, for example, or indicating 
whether they found a passage useful or interesting. 



Critical Stance 



Students are asked to consider how the author crafted a text. This includes 
identifying stylistic devices such as mood and tone. 



These stances are not considered hierarchical or completely independent of each other, but 
are iterative. They provide a frame for generating questions and considering student 
performance at all levels. All students at all levels should be able to respond to reading 
selections from all of these orientations. What varies with students’ developmental and 
achievement levels is the amount of prompting or support needed for response, the 
complexity of the texts to which they can respond, and the sophistication of their answers. 

As shown in Figure 2, the overall average proficiency of fourth-grade public-school 
students from Arkansas on the NAEP reading scale was 212. This proficiency was lower 
than that of students across the nation (21 6). 1 8 There also was a tremendous range in 
student performance as shown by the percentiles of the distribution of reading proficiency 
in Arkansas presented in Table 3. The lowest performing 10 percent of the fourth graders 
from Arkansas had proficiency levels below 164 while the top 10 percent had proficiency 
levels above 256. 



18 Differences reported as significant are statistically different at the 95 percent confidence level. This means 
that with 95 percent confidence there is a real difference in the average reading proficiency between the two 
populations of interest. “About the same” means that no statistically significant difference was found at the 
95 percent confidence level. 
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FIGURE 2 



Fourth-Grade Public-School Students’ 
Average Reading Proficiency 
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The NAEP reading scale ranges from 0 to 500. The standard errors are presented in parentheses. With about 
95 percent confidence, the average reading proficiency for each population of interest is within ± 2 standard 
errors of the estimated mean (95 percent confidence interval, denoted by hH). If the confidence intervals for 
the populations do not overlap, there is a statistically significant difference between the populations. If they do 
overlap, the difference may or may not be statistically significant. Statistical tests comparing the two estimates 
must be conducted that use the standard error of the difference (see Appendix A for details). 
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TABLE 3 



Percentiles of Reading Proficiency 
for Fourth-Grade Public-School 
Students 



5th 

Percentile 


10th 

Percentile 


25th 

Percentile 


50th 

Percentile 


75th 

Percentile 


90th 

Percentile 


95th 

Percentile 





149 ( 2.2) 164(1.9) 188 ( 1.2) 214 ( 1.5) 237(1.0) 256 ( 1.6) 267 ( 1.9) 



149(3.1) 163(3.7) 188 ( 3.7) 214 ( 3.6) 236 ( 2.5) 256 ( 4.3) 268 ( 3.2) 

152(2.0) 168(1.7) 193(1.1) 218(1.4) 241 ( 1.4) 261 ( 1.9) 272 ( 1.6) 



The NAEP reading scale ranges from 0 to 500. The standard errors of the statistics appear in parentheses. It 
can be said with about 95 percent confidence that, for each population of interest, the value for the entire 
population is within ± 2 standard errors of the estimate for the sample. In comparing two estimates, one must 
use the standard error of the difference (see Appendix A for details). 
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LEVELS OF READING ACHIEVEMENT 

Average proficiency on the NAEP scale provides an overall depiction of students’ reading 
achievement; however, by itself, it does not describe what students know and are able to 
do, nor does it evaluate student performance against a standard. This report next presents 
a set of results based on applying the National Assessment Governing Board’s standards 
to student performance on the reading scale. 

When Congress established the National Assessment Governing Board (NAGB) in 1988 
to set policy for NAEP, it charged the board with “identifying appropriate achievement 
goals for each age and grade in each subject area to be tested under the National 
Assessment.” (Pub.L. 297-100, Section 3403 (a)(5)(B)(ii)). To carry out this responsibility, 
NAGB contracted with American College Testing (ACT) to undertake advisory and 
analytic functions that could assist the Board in forming its conclusions as to appropriate 
achievement levels to be used for evaluating the 1992 reading assessment results. 
Achievement levels are mappings of collective judgments about how students should 
perform onto the achievement scale. 19 Boundary points were developed for three 
achievement levels for each grade — Basic, Proficient, and Advanced. Performance at the 
Basic level denotes partial mastery of the knowledge and skills that are fundamental for 
proficient work at the fourth-grade level. The central level, called Proficient, represents 
solid academic performance at the fourth-grade level. Students reaching this level 
demonstrate competency over challenging subject matter and are well prepared for the next 
level of schooling. Achievement at the Advanced level signifies superior performance in the 
fourth grade. 

This report follows NAGB’s policy that achievement levels should be the primary and 
initial method of presenting the results of the 1992 Trial State Assessment. In this report, 
these achievement levels are applied to the 1992 data, showing the proportions of students 
that achieved the three achievement levels. 

Definitions of the three levels of reading achievement are given in Figure 3. Examples of 
items at the achievement levels are provided. The reading passage which accompanies these 
items can be found in Appendix B. It should be noted that constructed-response items 
occur at all levels of reading achievement. 



er|c 



9 Appendix C briefly describes the process of gathering expert judgments about Basic, Proficient, and 
Advanced performance — as defined by NAGB policy — on each reading item, combining the various 
judgments on the various items and mapping them onto the scale, and setting the scale score cutpoints for 
reporting purposes based on these levels. 
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| Levels of Reading Achievement at Grade 4 
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The following achievement-level descriptions focus on the interaction of the reader, the 
text, and the context. They provide some specific examples of reading behaviors that 
should be familiar to most readers of this document. The specific examples are not 
inclusive; their purpose is to help clarify and differentiate what readers performing at each 
achievement level should be able to do. While a number of other reading achievement 
indicators exist at every level, space and efficiency preclude an exhaustive listing. It should 
also be noted that the achievement levels are cumulative from Basic to Proficient to 
Advanced. One level builds on the previous levels such that knowledge at the Proficient 
level presumes mastery of the Basic level, and knowledge at the Advanced level presumes 
mastery at both the Basic and Proficient levels. 




Fourth-grade students performing at the Basic level should demonstrate an 

understanding of the overall meaning of what they read. When reading texts 

appropriate for fourth graders, they should be able to make relatively obvious 

re- 
connections between the text and their own experiences. * 



Specifically, when reading literary text, they should be able to tell what the story is generally about - providing 
details to support their understanding - and be able to connect aspects of the stories to their own experiences. 

When reading informational text, Basic-level fourth graders should be able to tell what the selection is generally 
about or identify the purpose for reading it; provide details to support their understanding; and connect ideas 
from the text to their background knowledge and experiences. 





Fourth-grade students performing at the Proficient level should be able to 


ImpllaiWl 


demonstrate an overall understanding of the text, providing inferential as well 




as literal information. When reading text appropriate to fourth grade, they 




should be able to extend the ideas in the text by making inferences, drawing 


(243) "i 


conclusions, and making connections to their own experiences. The connection 


1 1 f 


between the text and what the student infers should be clear. 



Specifically, when reading literary text, Proficient-level fourth graders should be able to summarize the story, 
draw conclusions about the characters or plot, and recognize relationships such as cause and effect. 

When reading informational text, Proficient-level students should be able to summarize the information and 
identify the author’s intent or purpose. They should be able to draw reasonable conclusions from the text, 
recognize relationships such as cause and effect or similarities and differences, and identify the meaning of the 
selection’s key concepts. 





Fourth-grade students performing at the Advanced level should be able to 


ADVANCED 


generalize about topics in the reading selection and demonstrate an awareness 


111 | : LEVEli!!| 


of how authors compose and use literary devices. When reading text 


f|!P : <275 )-lllS 


appropriate to fourth grade, they should be able to judge texts critically and, in 




general, give thorough answers that indicate careful thought. 



Specifically, when reading literary text, Advanced-level students should be able to make generalizations about 
the point of the story and extend its meaning by integrating personal and other reading experiences with the 
ideas suggested by the text. They should be able to identify literary devices such as figurative language. 

When reading informational text. Advanced-level fourth graders should be able to explain the author’s intent by 
using supporting material from the text. They should be able to make critical judgments of the text (including 

BEST COPY AVAILABLE its form and content) and explain their judgments clearly. 
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The three items that follow were selected to exemplify each of the three achievement levels 
at grade 4. These items are all based on the story “Sybil Sounds the Alarm,” which is 
shown in its entirety in Appendix B. This is an historical narrative demonstrating the 
purpose “reading for literary experience.” For the multiple-choice items, the correct answer 
is marked with an asterisk. For the short constructed-response item, the scoring guide is 
provided. Also shown is the percent correct (conditional p-value) for the students 
performing within the interval of the indicated level. 




Sybil’s father thought that she 

A. was obedient but forgetful 
*B. was courageous and a good rider 

C. could lead the troops against the British 

D. could easily become angry 



PROFICIENT LEVEL 


Percent Correct for Proficient Interval 




Nation 


90(3.0); 



The information about the statue and stamp helps to show that 
*A. people today recognize and respect Sybil’s bravery 

B. people were surprised that George Washington honored her 

C. the author included minor details 

D. heroes are honored more now than they were then 
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How does the author show the excitement and danger of Sybil’s ride? 



Acceptable 

Acceptable answers indicate at least one of the following: 

• that she showed how concerned Sybil’s parents were about letting Sybil ride; 

• told how Sybil felt during the ride and immediately afterward; 

• told how dangerous the ride was. 

For example: 

• By letting you know there might be soldiers waiting to stop her; 

• By using special words to make it feel dangerous; 

• By using details like her mouth was dry with fear; 

• The way she described how she acted and how she looked; 

• There’s a battle going on near her, and she had to ride off the trail because red 
coats would stop her at any cost. 

Unacceptable 
For example: 

• By saying she was riding a horse. 
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DESCRIBING FOURTH-GRADE STUDENTS’ PERFORMANCE AT THE 
ACHIEVEMENT LEVELS 

Because the process of setting the levels of reading achievement centered on the descriptions 
of what students should be able to do, it is important to explore whether students actually 
met the expectations for performance at the Basic, Proficient, and Advanced levels. To 
help in this process, NCES arranged for ETS to apply a modified anchoring procedure to 
the 1992 reading achievement levels. A committee of reading education experts was 
assembled to review the questions and assessment results. Using their knowledge of reading 
and student performance on the individual questions, the committee members were asked 
to summarize student performance at each achievement level (see Appendix D for more 
details on the anchoring procedure). 

Placing the descriptions of how students performed at each of the levels in the context of 
the expectations for achievement at each of the levels and cross-checking with the actual 
question-by-question results yields some interesting findings. In general, the sets of reading 
skills expected were those observed. However, in some instances, particularly for extended 
response questions, even Advanced-level students had difficulty providing in-depth answers. 
In some other instances, because the assessment was developed prior to the achievement 
level descriptions, particular reading skills were not measured. 

In the description of students' performance beginning on the following page, each of the 
three achievement levels is discussed in turn — Basic, Proficient, then Advanced. For each 
of the three levels, the operational definition is presented (reproduced from Figure 3) 
followed in turn by a description of assessment performance at that achievement level 
which draws on the anchoring results. These descriptions are intended to be cumulative 
from Basic-level performance through Advanced. Therefore, demonstrated ability at the 
Proficient level presumes Basic-level performance, and Advanced performance presumes 
Proficient, as well as Basic abilities. 
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Fourth-grade students performing at the Basic level should demonstrate an 
understanding of the overall meaning of what they read . When reading texts 
appropriate for fourth graders, they should be able to make relatively obvious 
connections between the text and their own experiences . 



Specifically, when reading literary text, they should be able to tell what the story is generally about -- providing 
details to support their understanding - and be able to connect aspects of the stories to their own experiences. 

When reading informational text, Basic-level fourth graders should be able to tell what the selection is generally 
about or identify the purpose for reading it; provide details to support their understanding; and connect ideas 
from the text to their background knowledge and experiences. 

Fourth-grade students at the Basic level in the 1992 NAEP reading assessment were able 
to read uncomplicated narratives with understanding. The literary texts at this level 
included fables and realistic fiction about familiar topics. In addition, they were able to gain 
information from high-interest informative texts that were structured as narratives and dealt 
with relatively familiar topics, such as animals and sports. 

When reading literary text , Basic-level students demonstrated a general understanding of 
the stories by identifying an obvious theme or message. They answered questions about 
specific parts of the stories and provided details to support their understanding of 
characters’ feelings or actions. Fourth graders at the Basic level had considerable success 
in answering questions about the traits and functions of characters. For example, in the 
nation, 76 percent of the students within the Basic-level interval correctly answered the item 
about Sybil’s father. In addition, connections to their own experiences tended to involve 
aspects of characters. They could relate to the feelings of familiar characters. 

When reading informational text , Basic-level fourth graders were able to search for and 
locate explicit information in order to provide a summarization of part of the text. They 
were able to identify situations described in text and build simple inferences based on 
specific details. Although fourth-grade students were not asked directly to identify the 
purpose for reading an informational text, they were able to construct their own simple 
questions related to material they had read. They were only partially successful at making 
connections to background knowledge or experiences when reading to gain information. 



BEST COPY AVAILABLE 




THE 1992 NAEP TRIAL STATE ASSESSMENT 



35 



Arkansas 





Fourth-grade students performing at the Proficient level should be able to 


PROFICIENT 


demonstrate an overall understanding of the text, providing inferential as well 


' V £ j[ 


as literal information. When reading text appropriate to fourth grade, they 




should be able to extend the ideas in the text by making inferences, drawing 




conclusions, and making connections to their own experiences. The connection 
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between the text and what the student infers should be clear. 



Specifically, when reading literary text, Proficient-level fourth graders should be able to summarize the story, 
draw conclusions about the characters or plot, and recognize relationships such as cause and effect. 

When reading informational text, Proficient-level students should be able to summarize the information and 
identify the author’s intent or purpose. They should be able to draw reasonable conclusions from the text, 
recognize relationships such as cause and effect or similarities and differences, and identify the meaning of the 
selection’s key concepts. 

Fourth-grade students at the Proficient level were able to understand and extend the 
meaning of more difficult, unfamiliar literary pieces — those in culturally different or 
historical settings. They were able also to gain information, interpret meaning, and connect 
to background experiences when reading informative text that contained narrative elements 
and direct quotes. 

When reading literary text , Proficient-level fourth graders demonstrated an overall 
understanding by constructing responses to a story as a whole, as well as considering 
subtleties in aspects of stories. However, they were unable to provide an adequate story 
summary when asked to describe the major events in an historical fiction. Building on the 
skills demonstrated at the Basic level related to identifying and interpreting characters’ 
actions and feelings, Proficient-level students were able to draw conclusions about 
characters’ actions and recognize multiple character perspectives. In addition, they could 
recognize obvious cause-and-effect relationships that were related to story events. 
Fourth-grade students at this level demonstrated an ability to connect information in the 
story to the author’s purpose. For the example item, in the nation, 90 percent of the 
students within the Proficient-level interval were able to identify the significance of the 
information about the statue and the stamp in recognizing Sybil’s bravery. 

When reading informational text, Proficient -level fourth graders were able to identify major 
ideas and make straightforward inferences that were connected clearly to the text. They 
were able to recognize an author’s basic organizational pattern and general purpose. They 
could draw conclusions about key concepts and generalize across parts of the text. 
However, when asked to describe cause-and-effect relationships requiring a thoughtful 
consideration of implicit information, they were only partially successful. Their responses 
provided evidence that they could search for, locate, prioritize, and apply relevant 
information. Also, they could relate information from the selection to their own 
background experience and to inferences that were provided for them. 
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Fourth-grade students performing at the Advanced level should be able to 


M>VANCED.U 


generalize about topics in the reading selection and demonstrate an awareness 




of how authors compose and use literary devices. When reading text 




appropriate to fourth grade, they should be able to judge texts critically and , in 
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general, give thorough answers that indicate careful thought. 



Specifically, when reading literary text, Advanced-level students should be able to make generalizations about 
the point of the story and extend its meaning by integrating personal and other reading experiences with the 
ideas suggested by the text. They should be able to identify literary devices such as figurative language. 

When reading informational text, Advanced-level fourth graders should be able to explain the author’s intent by 
using supporting material from the text. They should be able to make critical judgments of the text (including 
its form and content) and explain their judgments clearly. 

Fourth-grade students at the Advanced level experienced success with literary and 
informative texts about less familiar topics. They not only demonstrated understanding 
of what they read, but also were able to extend, elaborate on, and examine the meaning 
of literary and informative text. 

When reading literary text , Advanced-level fourth graders were able to construct responses 
to a story and generalize about topics in a reading selection by selecting relevant 
information and building their own interpretations that remained consistent with the text. 
In addition, they were able to provide brief summarizations across the whole story. They 
demonstrated only partial ability, however, in integrating their personal experiences and 
other reading with ideas suggested by the text. Fourth graders at the Advanced level were 
able to understand some literary devices, such as figurative language, and could interpret 
authors’ intentions. For example, in the nation, 84 percent of the fourth -grade students 
within the Advanced-level interval were able to provide acceptable responses to the 
question about the author’s techniques in the story about Sybil. 

When reading informational text , Advanced-level students were able to provide an 
explanation of the author’s techniques for presenting information, although fourth graders 
were not explicitly asked to support their explanations. They did, however, use 
information presented in the text to answer other questions. For example, they were able 
to make critical judgments about the form and content of the text by indicating the relative 
importance of ideas and were able to gain a more thorough understanding of a particular 
topic. Some Advanced-level fourth graders could develop their own ideas based on the 
information presented in the passages and form more complex questions about a selection. 
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Figure 4 provides the percentage of students at or above each achievement level, as well 
as the percentage of students below the Basic level. In Arkansas, 53 percent of the fourth 
graders in public schools were at or above the Basic level, 20 percent were at or above the 
Proficient level, and 3 percent were at or above the Advanced level. Nationwide, 

57 percent of the fourth graders were at or above the Basic level, 24 percent were at or 
above the Proficient level, and 4 percent were at or above the Advanced level. A 
somewhat smaller percentage of students in Arkansas than across the nation were at or 
above the Proficient level. 



FIGURE 4 



Levels of Fourth-Grade Public-School 
Students’ Reading Achievement 
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3 ( 0 . 4 ) 
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20 ( 1 . 3 ) 
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24 ( 1 . 2 ) 



53 ( 1 . 6 ) 
52 ( 3.5) 
57 ( 1 . 2 ) 



47 ( 1 . 6 ) 

48 ( 3.5) 
43 ( 1 . 2 ) 



PERCENT 

The standard errors are presented in parentheses. With about 95 percent confidence, the 
average reading proficiency for each population of interest is within ± 2 standard errors of 
the estimated mean (95 percent confidence interval, denoted by H4H). If the confidence 
intervals for the populations do not overlap, there is a statistically significant difference 
between the populations. If they do overlap, the difference may or may not be statistically 
significant. Statistical tests comparing the two estimates must be conducted that use the 
standard error of the difference (see Appendix A for details). 
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Clearly, many students in Arkansas fail to meet or exceed the achievement levels that 



policymakers will need to look to many sources of information and opinion for 
explanations of these levels of performance. Among the possible explanations, several 
factors should not be overlooked. First, students may not be learning enough in school to 
reach the achievement levels. In 1983, the National Commission on Excellence in 
Education warned that “the educational foundations of our society are being eroded by a 
rising tide of mediocrity that threatens our very future.” 20 In 1990, the President and the 
Governors committed the Nation to six goals for education, the third of which called for 
American students to “leave grades four, eight and twelve having demonstrated competency 
in challenging subject matter.” Many political leaders of this nation have expressed 
dissatisfaction with the performance of American students. These NAEP findings confirm 
that a great many American students are not yet performing at high levels. 

Second, some students may not be reaching the higher achievement levels because schools 
may not be teaching the elements of reading that are included on the NAEP assessment, 
and because the assessment may not be covering some elements of reading included in the 
school curriculum. No assessment or test can cover all the different areas of reading that 
are taught in school. The content coverage of the NAEP reading assessment was set by a 
consensus approach. Teachers, curriculum specialists, subject matter specialists, local 
school administrators, parents, and members of the general public actively participated in 
deciding what are the most important elements of reading to be included in the assessment 
and for students to learn. 21 

Third, the Basic, Proficient, and Advanced achievement levels reflect high performance 
standards for the 1992 NAEP reading scale. The establishment of achievement levels 
depends on securing a set of informed judgments of expectations for student educational 
performance and on summarizing the individual ratings into collective judgments. These 
expectations reflect the Board’s policy definitions, which require that students at the central, 
Proficient level demonstrate “competency over challenging subject matter.” The resulting 
standards are rigorous. 

As measures of performance, both average proficiency scores and percentages of students 
who score at or above the critical achievement levels on the NAEP scale provide a valuable 
overall depiction of students’ reading achievement. In order to present a closer look at how 
well students know particular areas of reading, the next section presents student 
performance according to two purposes for reading. 



20 National Commission on Excellence in Education, A Nation at Risk. (Washington, DC: U.S. Department 
of Education, 1983). In 1988, then-Secretary Bennett reported that the “precipitous downward slide of 
previous decades has been arrested, and we have begun the long climb back to reasonable standards.” (p. 1 
in American Education: Making it Work. (Washington, DC: U.S. Department of Education, 1988).) 

21 NAEP Reading Consensus Project. Reading Framework for the 1992 National Assessment of Educational 
Progress. (Washington, DC: National Assessment Governing Board, 1992). 
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PERFORMANCE ACCORDING TO PURPOSE FOR READING 



As previously indicated, the cognitive questions in the Trial State Assessment covered two 
purposes for reading at grade 4 — reading for literary experience and reading to gain 
information. Figure 5 (average proficiency) and Table 4 (percentiles) provide results for 
Arkansas, the Southeast region, and the nation according to each reading purpose. 
Students in Arkansas performed lower than students across the nation in reading for literary 
experience. 



FIGURE 5 



Fourth-Grade Public-School Students’ 
Average Reading Proficiency According to 
Purpose for Reading 
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The NAEP reading scale ranges from 0 to 500. The standard errors are presented in 
parentheses. With about 95 percent confidence, the average reading proficiency for each 
population of interest is within ± 2 standard errors of the estimated mean (95 percent 
confidence interval, denoted by hH). If the confidence intervals for the populations do not 
overlap, there is a statistically significant difference between the populations. If they do 
overlap, the difference may or may not be statistically significant. Statistical tests comparing 
the two estimates must be conducted that use the standard error of the difference (see 
Appendix A for details). 
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Percentiles of Reading Proficiency 
for Fourth-Grade Public-School 
Students by Purpose for Reading 



5th 


10th 


25th 
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Percentile 


Percentile 


Percentile 


Percentile 


Percentile 


Percentile 


Percentile 



Reading for 

Literary 

Experience 

Arkansas 

Southeast 

Nation 

Reading to Oain 
Information 

Arkansas 

Southeast 

Nation 



||| 

145 ( 2.2) 161 ( 1.9) 187 ( 1.9) 216 ( 1.5) 241 j 1.7) 



148 < 3.3) 163 ( 3.6) 189 ( 3.8) 216 ( 2.8) 239 ( 2.7) 259 ( 2.3) 271 ( 4.1) 




The NAEP reading scale ranges from 0 to 500. The standard errors of the statistics appear in parentheses. It 
can be said with about 95 percent confidence that, for each population of interest, the value for the entire 
population is within ± 2 standard errors of the estimate for the sample. In comparing two estimates, one must 
use the standard error of the difference (see Appendix A for details). 
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CHAPTER 2 

Reading Performance by Subpopulations 



Assessment results repeatedly show differences in achievement for subpopulations of 
students. 22 The 1992 Trial State Assessment provides additional information about the 
achievement of important subpopulations by reporting on the performance of various 
subgroups of the student population defined by race/ethnicity, type of community, parents’ 
education level, and gender. 



RACE/ETHNICITY 

The Trial State Assessment results for different racial/ethnic groups can be compared when 
the number of students in a racial/ethnic group is sufficient in size to be reliably reported 
(at least 62 students). Figure 6 (average proficiency) and Table 5 (percentiles) present 
reading performance results for White, Black, Hispanic, and American Indian students from 
Arkansas. 



qOPY ML ABLE 



22 Ina V.S. Mullis, John A. Dossey, Mary A. Foertsch, Lee R. Jones, and Claudia A. Gentile. 
Academic Progress. (Washington, DC: National Center for Education Statistics, 1992). 
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As shown in Figure 6, White students in Arkansas demonstrated higher average reading 
proficiency than did Black, Hispanic, or American Indian students. 



FIGURE 6 



Fourth-Grade Public-School Students’ 
Average Reading Proficiency by 
Race/Ethnicity 
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The NAEP reading scale ranges from 0 to 500. The standard errors are presented in parentheses. With about 
95 percent confidence, the average reading proficiency for each population of interest is within ± 2 standard 
errors of the estimated mean (95 percent confidence interval, denoted by HH). If the confidence intervals for 
the populations do not overlap, there is a statistically significant difference between the populations. If they do 
overlap, the difference may or may not be statistically significant. Statistical tests comparing the two estimates 
must be conducted that use the standard error of the difference (see Appendix A for details). ! Interpret with 
caution - the nature of the sample does not allow accurate determination of the variability of this statistic. 
*** Sample size is insufficient to permit a reliable estimate (fewer than 62 students). 
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Percentiles of Reading Proficiency 
for Fourth-Grade Public-School 
Students by Race/Ethnicity 
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The NAEP reading scale ranges from 0 to 500. The standard errors of the statistics appear in parentheses. It 
can be said with about 95 percent confidence that, for each population of interest, the value for the entire 
population is within ± 2 standard errors of the estimate for the sample. In comparing two estimates, one must 
use the standard error of the difference (see Appendix A for details). *** Sample size is insufficient to permit 
a reliable estimate (fewer than 62 students). 



Figure 7 provides the percentage of students by race/ethnicity group at or above each of 
the three achievement levels and also the percentage below the Basic level. In Arkansas, 
about one quarter of the White students (25 percent), relatively few of the Black students 
(5 percent), relatively few of the Hispanic students (6 percent), and some of the American 
Indian students ( 14 percent) were at or above the Proficient level. Across the nation, about 
one quarter of the White students (30 percent), relatively few of the Black students 
(7 percent), some of the Hispanic students (12 percent), and some of the American Indian 
students (13 percent) were at or above the Proficient level. 
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FIGURE 7 



Levels of Fourth-Grade Public-School 
Students’ Reading Achievement by 
Race/Ethnicity 
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PERCENT 

The standard errors are presented in parentheses. With about 95 percent confidence, the 
average reading proficiency for each population of interest is within ± 2 standard errors of 
the estimated mean (95 percent confidence interval, denoted by MH). If the confidence 
intervals for the populations do not overlap, there is a statistically significant difference 
between the populations. If they do overlap, the difference may or may not be statistically 
significant. Statistical tests comparing the two estimates must be conducted that use the 
standard error of the difference (see Appendix A for details). ! Interpret with caution — the 
nature of the sample does not allow accurate determination of the variability of this statistic. 
*** Sample size is insufficient to permit a reliable estimate (fewer than 62 students). 
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TYPE OF COMMUNITY 

Figure 8 (average proficiency) and Table 6 (percentiles) present the reading proficiency 
results for fourth-grade students attending public schools in areas classified as “other”, 
disadvantaged urban areas, and extreme rural areas. (These are the “type of community” 
groups in Arkansas with student samples large enough to be reliably reported.) The results 
indicate that the average reading performance of Arkansas students attending schools in 
areas classified as “other” was higher than that of students attending schools in 
disadvantaged urban areas and about the same as that of students attending schools in 
extreme rural areas. 



FIGURE 8 



Fourth-Grade Public-School Students’ 
Average Reading Proficiency by Type of 
Community 
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The NAEP reading scale ranges from 0 to 500. The standard errors are presented in parentheses. With about 
95 percent confidence, the average reading proficiency for each population of interest is within ± 2 standard 
errors of the estimated mean (95 percent confidence interval, denoted by l-M). If the confidence intervals for 
the populations do not overlap, there is a statistically significant difference between the populations. If they do 
overlap, the difference may or may not be statistically significant. Statistical tests comparing the two estimates 
must be conducted that use the standard error of the difference (see Appendix A for details). ! Interpret with 
caution — the nature of the sample does not allow accurate determination of the variability of this statistic. 
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Percentiles of Reading Proficiency 
for Fourth-Grade Public-School 
Students by Type of Community 
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The NAEP reading scale ranges from 0 to 500. The standard errors of the statistics appear in parentheses. It 
can be said with about 95 percent confidence that, for each population of interest, the value for the entire 
population is within ± 2 standard errors of the estimate for the sample. In comparing two estimates, one must 
use the standard error of the difference (see Appendix A for details). 



Figure 9 presents reading performance by achievement levels. Some of the students 
attending schools in areas classified as “other” (20 percent), some of the students in 
disadvantaged urban areas (12 percent), and some of the students in extreme rural areas 
(18 percent) in Arkansas were at or above the Proficient level. Across the nation, about 
one quarter of the students in areas classified as “other” (24 percent), relatively few of the 
students in disadvantaged urban areas (5 percent), and about one quarter of the students 
in extreme rural areas (24 percent) were at or above the Proficient level. 
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FIGURE 9 



Levels of Fourth-Grade Public-School 
Students’ Reading Achievement by Type of 
Community 
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Levels of Fourth-Grade Public-School 
Students’ Reading Achievement by Type of 
Community 
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38 ( 4.0) 
42 ( 1.4) 



PERCENT 

The standard errors are presented in parentheses. With about 95 percent confidence, the 
average reading proficiency for each population of interest is within ± 2 standard errors of 
the estimated mean (95 percent confidence interval, denoted by l-M). If the confidence 
intervals for the populations do not overlap, there is a statistically significant difference 
between the populations. If they do overlap, the difference may or may not be statistically 
significant. Statistical tests comparing the two estimates must be conducted that use the 
standard error of the difference (see Appendix A for details). ! Interpret with caution - the 
nature of the sample does not allow accurate determination of the variability of this statistic. 



PARENTS’ EDUCATION LEVEL 

Previous NAEP findings have shown that students whose parents are better educated tend 
to have higher reading proficiency. Figure 10 (average proficiency), Table 7 (percentiles), 
and Figure 1 1 (achievement levels) show the reading performance results for fourth-grade 
public-school students who reported that at least one parent graduated from college, at least 
one parent had some education after high school, at least one parent graduated from high 
school, neither parent graduated from high school, or they did not know their parents’ 
education level. Note that a substantial percentage of fourth-grade students did not know 
their parents’ education level. 
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As shown in Figure 10, students in Arkansas who reported that at least one parent 
graduated from college demonstrated about the same average reading proficiency as did 
students who reported that at least one parent had some education after high school or at 
least one parent graduated from high school, but higher proficiency than did students who 
reported that neither parent graduated from high school or they did not know their parents’ 
education level. 



FIGURE 10 



Fourth-Grade Public-School Students’ 
Average Reading Proficiency by Parents’ 
Level of Education 
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The NAEP reading scale ranges from 0 to 500. The standard errors are presented in parentheses. With about 
95 percent confidence, the average reading proficiency for each population of interest is within ± 2 standard 
errors of the estimated mean (95 percent confidence interval, denoted by HH). If the confidence intervals for 
the populations do not overlap, there is a statistically significant difference between the populations. If they do 
overlap, the difference may or may not be statistically significant. Statistical tests comparing the two estimates 
must be conducted that use the standard error of the difference (see Appendix A for details). 
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TABLE 7 



Percentiles of Reading Proficiency 
for Fourth-Grade Public-School 
Students by Parents’ Level of 
Education 
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Percentile 
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Nation 
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HS graduate 
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Southeast 
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Southeast 

Nation 

I don’t know 

Arkansas 

Southeast 

Nation 



155 ( 3.7) 169 ( 2.2) 194 ( 4.9) 221(2.0) 244 ( 2.3) 263 ( 2.7) 273(1.8) 

246 ( 45) 

160 ( 2.7) 175 ( 4.0) 200 ( 2.4) 227 ( 2.5) 250 ( 1.5) 269 ( 1.3) 280 ( 2.8) 

168 ( 3.3) 180 ( 3.0) 202 ( 3.6) 227 ( 2.4) ' 247:^3)1)11 263T3)9) * 272 (j3J0) 

150(10.0) 167 ( 8.3) 195 (>.$)& 217 ( 4.0) 2©>j 861 

( 6.0) 202 ( 3.8) 225 ( 4.4) 246 ( 3.?)fg26g;(^||)'»|:277 ( 4.9) 

149 ( 9.2) 165 ( 3.5) 189 ( 2.3) 215 ( 3.3) 237 ( 1.7) 254 ( 2.5) 265 ( 2.0) 

151 (12.0) 160(5.9) 

151 ( 3.4) 165 ( 1.0) 190 (;2|pg5|l5,( 1.9) 236 ( 3.0) i'254|j2,4) ”265 (3,1). 

147 ( 5.7) 158 ( 8.9) 181 ( 3.4) 203 i 3.3) 224 ( 2.5) 247 ( 5.2 ) 257 ( 3.5) 

146"f5^) ! 155(10.1) 175(10.3) 199 (4^)|Z221|(:6.8) : 237 ( 7.0) 251 ( 8.5) 

142 ( 5.3) 154 ( 64) 175 ( 8.2) 199 ( 3.0) 222 ( 6.4) 243 ( 4.8) 255 ( 9.8) 
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The NAEP reading scale ranges from 0 to 500. The standard errors of the statistics appear in parentheses. It 
can be said with about 95 percent confidence that, for each population of interest, the value for the entire 
population is within ± 2 standard errors of the estimate for the sample. In comparing two estimates, one must 
use the standard error of the difference (see Appendix A for details). 



Further, from Figure 1 1 , reading achievement in Arkansas was at or above the Proficient 
level for 26 percent of the students who reported that at least one parent graduated from 
college, 30 percent of the students who reported that at least one parent had some 
education after high school, 19 percent of the students who reported that at least one 
parent graduated from high school, 12 percent of the students who reported that neither 
parent graduated from high school, and 12 percent of the students who reported that they 
did not know their parents’ education level. Across the nation, these figures were 
33 percent of the students who reported that at least one parent graduated from college, 

28 percent of the students who reported that at least one parent had some education after 
high school, 18 percent of the students who reported that at least one parent graduated 
from high school, 10 percent of the students who reported that neither parent graduated 
from high school, and 17 percent of the students who reported that they did not know their 
parents’ education level. 
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Level of Education 
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PERCENT 

The standard errors are presented in parentheses. With about 95 percent confidence, the 
average reading proficiency for each population of interest is within ± 2 standard errors of 
the estimated mean (95 percent confidence interval, denoted by HH). If the confidence 
intervals for the populations do not overlap, there is a statistically significant difference 
between the populations. If they do overlap, the difference may or may not be statistically 
significant. Statistical tests comparing the two estimates must be conducted that use the 
standard error of the difference (see Appendix A for details). 
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GENDER 



In general, NAEP reading assessment results for males and females support numerous 
studies that have revealed gender differences favoring females in reading. 23 The 1992 Trial 
State Assessment results for Arkansas support those findings. 



As shown in Figure 12, in Arkansas, fourth-grade boys attending public schools had a 
lower average reading proficiency than did fourth-grade girls. Compared to the national 
results, girls in Arkansas performed lower than girls across the country; boys in Arkansas 
performed about the same as boys across the country. Table 8 provides the percentiles for 
fourth-grade reading performance results by gender. 



FIGURE 12 



Fourth-Grade Public-School Students’ 
Average Reading Proficiency by Gender 
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The NAEP reading scale ranges from 0 to 500. The standard errors are presented in parentheses. With about 
95 percent confidence, the average reading proficiency for each population of interest is within ± 2 standard 
errors of the estimated mean (95 percent confidence interval, denoted by HH). If the confidence intervals for 
the populations do not overlap, there is a statistically significant difference between the populations. If they do 
overlap, the difference may or may not be statistically significant. Statistical tests comparing the two estimates 
must be conducted that use the standard error of the difference (see Appendix A for details). 



23 Gita Z. Wilder and Kristin Powell, Sex Differences in Test Performance: A Survey of the Literature. (New 
York: College Entrance Examination Board, 1989). 
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TABLE 8 



Percentiles of Reading Proficiency 
for Fourth-Grade Public-School 
Students by Gender 
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The NAEP reading scale ranges from 0 to 500. The standard errors of the statistics appear in parentheses. It 
can be said with about 95 percent confidence that, for each population of interest, the value for the entire 
population is within ± 2 standard errors of the estimate for the sample. In comparing two estimates, one must 
use the standard error of the difference (see Appendix A for details). 



As shown in Figure 13, there was a significant difference between the percentages of males 
and females in Arkansas who attained the Proficient level (22 percent for females and 
17 percent for males). The percentage of females in Arkansas who attained the Proficient 
level was about the same as the percentage of females in the nation who attained the 
Proficient level (22 percent for Arkansas and 26 percent for the nation). However, the 
percentage of males in Arkansas who attained the Proficient level was somewhat lower than 
the percentage of males in the nation who attained the Proficient level (17 percent for 
Arkansas and 21 percent for the nation). 



PERFORMANCE ACCORDING TO PURPOSE FOR READING 

Table 9 provides a summary of performance according to each of the two purposes for 
reading by race/ethnicity, type of community, parents’ education level, and gender. 
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The standard errors are presented in parentheses. With about 95 percent confidence, the 
average reading proficiency for each population of interest is within ± 2 standard errors of 
the estimated mean (95 percent confidence interval, denoted by HH), If the confidence 
intervals for the populations do not overlap, there is a statistically significant difference 
between the populations. If they do overlap, the difference may or may not be statistically 
significant. Statistical tests comparing the two estimates must be conducted that use the 
standard error of the difference (see Appendix A for details). 




58 



THE 1992 NAEP TRIAL STATE ASSESSMENT 



Arkansas 



NATION’S 

rasp I 



THE 
REPORT 
CARD 

1992 

Trtal State Assessment 




TABLE 9 



Fourth-Grade Public-School 
Students’ Average Reading 
Proficiency for “Purpose for 
Reading” by Subpopulaiion 



Reading for 


Reading to Gain 


Literary Experience 


Information 



TOTAL 




Proficiency ^ s ' 




Proficiency 5 




Arkansas 


‘ ,'f? 




- , -2l1ol^l-S)'..' S *■' 




Southeast 


t & Y ■; . :'2lSTf*2;5) ” \ 




1 12® (§2.7) •, . •• - 




Nation 






213(1.2) 


RACE/ETHNICITY 








White 


Arkansas 


■r's 222 ( 1.3) "S. ; 




y "P 219 (1.3) : i 




Southeast 




> 






Nation 




' 


222^0i6}^fi|S| 


Black 


Arkansas 






Mil Wt: ~$ 2 M 




Southeast 


pjgap jgt 




PMWP(I2:8) ' 




Nation 


in?) 7 




189 ( 1.9) 


Hispanic 


Arkansas 










Southeast 






194(i3(5'!M^^g 




Nation 


205 ( 2.6) 






American Indian 


Arkansas 






^fc20|fi4.9 ) .■>: !g 1 




Southeast 










Nation 


209 ( 5.0) 




202 ( 5.2) \ 


TYPE OF COMMUNITY 




.::Ik ■ i|| p I ||§ i|| §iig||t| # 






Disadvantaged urban 


Arkansas 


200(4.9)! 




194 ( 7.3)! 




Southeast 


188 ( 3.0)! 


, 


' . 186'p3BB)liS8ii|^ 




Nation 


V; i 191 ( 2.7) , 




, leap^WpiaMi 


Extreme rural 


Arkansas 


: : :: 214 ( 3.5) 








Southeast 


tiff ' 2l5t;5.4)!.i; : fcs 




^^21 0?('SI1;)fef5f%*S 




Nation 


222 ( 2.9) 




■ 218'('3.4)'P'P 1 1 


Other 


Arkansas 


214 ( 1.9) 


■ 


211 ( 1.8) 




Southeast 


216(3.2) 




3;i 




Nation 


219,(1.4). v : . ■ , 


. 


p Y 214 (1.4) | r 


PARENTS’ EDUCATION 




Y :P ’ : i: : ; v :Y; Y ; 




i- ; V: gpi PpPi*-: '*.■ vC<V< ■ r b 


College graduate 


Arkansas 


-i. ' Y ■ 218 ( 2.3) : P 


1111 


®tM^I'8|(!2.2)fM^?« 




Southeast 










Nation 


226 ( 1.6) 






Some after HS 


Arkansas 










Southeast 


. 219(5.5) | . 








Nation 






fi8#S2l9 (?2;Z) 


HS graduate 


Arkansas 






lira&W) “ p % 




Southeast 






“fe,f*205?(-4T6),T^,¥i 




Nation 






P 208 p2.0) 


HS non-graduate 


Arkansas 


®^W2"04S( ; 3.0) 


" V 


pfnHoi^( 




Southeast 


Hpfe‘^201 ('3';8):f»: ; %Xfi'Y. 


‘ 


(-4.4)' <£l ■- 




Nation 








1 don’t know 


Arkansas 






fP%202.( 2.1) 




Southeast 


207 ( 2.9) . 








Nation 


- 212 ( 1.4) • 




207 ( 1.5) 


GENDER 




ill || - § || V-: Y pi illpli 


■ 1 


Male 


Arkansas 


209 ( 




208 ( 1.7) 




Southeast 


a-: 206 Ll2) Pixii Px 




P 204.t'2i9)&:'X,^# 




Nation 






Pi-’ X' 210.( li5)S|ipiS^. 


Female 


Arkansas 






^ft2i’iS(Cp)S^ P: 




Southeast 


220;(2:4) v-iF. pi :-V; 




214?(f2.9)' PSSPi 




Nation 


223 ( 1.1) 




216(1.4) 



The NAEP reading scale ranges from 0 to 500. The standard errors of the statistics appear in parentheses. It 
can be said with about 95 percent confidence that, for each population of interest, the value for the entire 
population is within ± 2 standard errors of the estimate for the sample. In comparing two estimates, one must 
use the standard error of the difference (see Appendix A for details). ! Interpret with caution - the nature of 
the sample does not allow accurate determination of the variability of this statistic. *** Sample size is 
insufficient to permit a reliable estimate (fewer than 62 students). 
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PART TWO 



THE NATION’S 
REPORT 
CARD 



1992 



Trial State Assessment 




Finding a Context for Understanding Students’ 
Reading Proficiency 



Information on the reading performance of students in Arkansas can be better understood 
and used for improving instruction and setting policy when supplemented with contextual 
information about schools, teachers, and students. 

To gather contextual information, the fourth-grade students participating in the 1992 Trial 
State Assessment, their reading teachers, and the principals or other administrators in their 
schools were asked to complete questionnaires on policies, instruction, and programs. The 
student, teacher, and school data help to describe some of the current practices and 
emphases in reading education, illuminate some of the factors that appear to be related to 
fourth-grade public-school students’ reading proficiency, and provide an educational 
context for understanding information on student achievement. 

It is important to note that the NAEP data cannot establish cause-and-effect links between 
various contextual factors and students’ reading proficiency. However, the results do 
provide information about important relationships between the contextual factors and 
proficiency. Through the questionnaires administered to students, teachers, and principals, 
NAEP is able to provide a broad picture of educational practices prevalent in American 
schools and classrooms. 

Part Two consists of four chapters. Chapter 3 discusses policies and practices related to 
reading. Chapter 4 focuses on instructional practices — how instruction is delivered. 
Chapter 5 provides information about teachers, and Chapter 6 examines students’ home 
support for literacy. 
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CHAPTER 3 

Policies and Practices Related to Reading 

This chapter focuses on curricular and instructional content issues in Arkansas public 
schools and their relationship to students’ reading proficiency. Table 10 provides a profile 
of the reading policies in the public schools with fourth grades in Arkansas. Some of the 
selected results obtained from teacher and school questionnaires reveal: 



• According to the schools in Arkansas, 80 percent of the fourth-grade 
students were in schools where reading was identified as receiving special 
emphasis. This compares with 86 percent across the country. 

• According to their reading teachers, 45 percent of the students in Arkansas 
were typically taught reading in a class that was grouped by reading ability. 
Ability grouping was about as prevalent across the nation (34 percent). 

• According to the schools in Arkansas, 38 percent of the students were in 
schools in which the fourth-grade students stay with the same teacher for 
all academic subjects, 17 percent were in schools in which students have 
different teachers in most or all academic subjects, and 45 percent were in 
schools in which students remain with one teacher for most subjects but 
may have a different teacher for one or two subjects. Across the country 
these figures were 48 percent, 10 percent, and 42 percent, respectively. 

• According to the teachers in Arkansas, 48 percent of the students had 
teachers who had a reading curriculum specialist available to help or advise. 
Nationally, 64 percent of the students had teachers who had a reading 
curriculum specialist available to help or advise. 

• According to the schools in Arkansas, 77 percent of the fourth graders were 
in schools in which parents were used as aides in the classroom. This 
compares with 89 percent across the country. 
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TABLE 10 



Reading Policies and Practices in 
Arkansas Fourth-Grade Public 
Schools 
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The standard errors of the statistics appear in parentheses. It can be said with about 95 percent confidence that, 
for each population of interest, the value for the entire population is within ± 2 standard errors of the estimate 
for the sample. In comparing two estimates, one must use the standard error of the difference (see 
Appendix A for details). 



TIME FOR INSTRUCTIONAL ACTIVITIES 

Studies of school effectiveness indicate that schools that are successful in teaching reading 
maximize the amount of time available for student learning. 24 Thus, to begin to place 
students’ reading proficiency in context, it is useful to examine the extent to which 
fourth-grade students’ reading teachers in Arkansas are spending their time on instructional 
activities. Students’ teachers were asked to report on the amount of time they spent with 
each class for reading instruction on a typical day. Table 1 1 and Table A1 1 (Page 136) in 
the Data Appendix 25 show that: 



24 R.C. Anderson, E.H. Hiebert, J.A. Scott, and I.A.G. Wilkinson. Becoming a Nation of Readers: The Report 
of the Commission on Reading. (U.S. Department of Education: The National Institute of Education, 1985). 



25 For every table in the body of the report that includes estimates of average proficiency, the Data Appendix 
provides a corresponding table presenting the results for the four subpopulations -- race /ethnicity, type of 
community, parents’ education level, and gender. 
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• In Arkansas, 55 percent of the fourth-grade students had reading teachers 
who spent at least 60 minutes providing reading instruction each day. By 
comparison, 45 percent of the students had reading teachers who spent 
45 minutes or less providing reading instruction each day. 

• Across the nation, 71 percent of the fourth-grade students had reading 
teachers who spent at least 60 minutes providing reading instruction each 
day. Additionally, 29 percent of the students had reading teachers who 
spent 45 minutes or less providing reading instruction each day. 

• In Arkansas, students whose reading teachers provided at least 60 minutes 
of reading instruction had about the same reading proficiency as did 
students whose teachers provided 45 minutes or less of reading instruction 
each day. 

• In Arkansas, the results by type of community show that 55 percent of the 
students attending schools in areas classified as “other”, 54 percent of the 
students in disadvantaged urban areas, and 55 percent of the students in 
extreme rural areas had teachers who spent at least 60 minutes providing 
reading instruction each day . 




Trial State Assessment 



TABLE 1 1 



Teachers’ Reports on Time Spent 
Teaching Reading 



Arkansas 



Southeast 



Nation 




The NAEP reading scale ranges from 0 to 500. The standard errors of the statistics appear in parentheses. It 
can be said with about 95 percent confidence that, for each population of interest, the value for the entire 
population is within ± 2 standard errors of the estimate for the sample. In comparing two estimates, one must 
use the standard error of the difference (see Appendix A for details). ! Interpret with caution — the nature of 
the sample does not allow accurate determination of the variability of this statistic. 
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INSTRUCTIONAL EMPHASIS 



There is no single best method for teaching and learning reading that is proven to be 
successful for everyone. Rather, it is likely that a variety of approaches and programs can 
produce able readers. To provide information about the major types of reading approaches 
used in Arkansas, fourth-grade students’ reading teachers were asked to report on the 
amount of instructional emphasis they gave to six methods for teaching reading — phonics, 
integration of reading and writing, whole language, literature-based reading, reading across 
the content areas, and individualized reading programs. 



Table 12 provides the results for the extreme emphasis categories — “heavy emphasis” and 
“little or no emphasis” -- for each of the six methods. According to the reading teachers: 

• In Arkansas, some of the fourth-grade students (15 percent) were being 
taught by teachers who placed heavy emphasis on phonics; less than half 
(31 percent) were being taught by teachers who placed heavy emphasis on 
the integration of reading and writing; and about one quarter (21 percent) 
were being taught by teachers who placed heavy emphasis on the whole 
language approach. 

• In addition, in Arkansas, about one quarter of the fourth-grade students 
(24 percent) were being taught by teachers who placed heavy emphasis on 
literature-based reading; less than half (39 percent) were being taught by 
teachers who placed heavy emphasis on reading across the content areas; 
and some (12 percent) were being taught by teachers who placed heavy 
emphasis on individualized reading programs. 

• By comparison, in Arkansas, about one quarter of the fourth-grade 
students (22 percent) were being taught by teachers who placed little or 
no emphasis on phonics; some (13 percent) were being taught by teachers 
who placed little or no emphasis on the integration of reading and writing; 
and about one quarter (29 percent) were being taught by teachers who 
placed little or no emphasis on the whole language approach. 

• In addition, in Arkansas, some of the fourth-grade students (20 percent) 
were being taught by teachers who placed little or no emphasis on 
literature-based reading; some (17 percent) were being taught by teachers 
who placed little or no emphasis on reading across the content areas; and 
about half (49 percent) were being taught by teachers who placed little or 
no emphasis on individualized reading programs. 
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TABLE 12 



Teachers’ Reports on Emphasis 
Given to Specific Methods for 
Teaching Reading 
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Teacher “emphasis” categories 
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The NAEP reading scale ranges from 0 to 500. The standard errors of the statistics appear in parentheses. It 
can be said with about 95 percent confidence that, for each population of interest, the value for the entire 
population is within ± 2 standard errors of the estimate for the sample. In comparing two estimates, one must 
use the standard error of the difference (see Appendix A for details). The percentages may not total 
100 percent because the “Moderate Emphasis” category is not included. ! Interpret with caution - the nature 
of the sample does not allow accurate determination of the variability of this statistic. *** Sample size is 
insufficient to permit a reliable estimate (fewer than 62 students). 



BEST COPT AVAILABLE 




73 



THE 1992 NAEP TRIAL STATE ASSESSMENT 



67 



Arkansas 



SUMMARY 



Effective teachers of reading create a literate classroom environment. They allocate an 
adequate amount of time to reading and writing, sustain children’s attention, maintain a 
brisk pace, and keep rates of success high. 26 In Arkansas, the information on curricular 
and instructional content issues has revealed the following: 

• According to the schools in Arkansas, 80 percent of the fourth-grade 
students were in schools where reading was identified as receiving special 
emphasis. This compares with 86 percent across the country. 

• According to the teachers in Arkansas, 48 percent of the students had 
teachers who had a reading curriculum specialist available to help or advise. 
Nationally, 64 percent of the students had teachers who had a reading 
curriculum specialist available to help or advise. 

• According to the schools in Arkansas, 77 percent of the students were in 
schools in which parents were used as aides in the classroom. This 
compares with 89 percent across the country. 

• In Arkansas, 55 percent of the fourth-grade students had reading teachers 
who spent at least 60 minutes providing reading instruction each day. By 
comparison, 45 percent of the students had reading teachers who spent 
45 minutes or less providing reading instruction each day. 



26 R.C. Anderson, E.H. Hiebert, J.A. Scott, and l.A.G. Wilkinson. Becoming a Nation of Readers: The Report 
of the Commission on Reading. (U.S. Department of Education: The National Institute of Education, 1985). 

74 



68 



THE 1992 NAEP TRIAL STATE ASSESSMENT 



Arkansas 



CHAPTER 4 

How Is Reading Instruction Delivered? 

Effective classroom instruction can help students become thoughtful readers. 27 The 
instructional activities that students engage in can also lead them to view reading in 
particular ways 28 and to focus on developing certain skills and strategies. To provide 
information about how instruction is delivered in Arkansas, fourth-grade public-school 
students participating in the Trial State Assessment Program and their reading teachers 
were asked to report on the use of various teaching and learning activities in their reading 
classrooms. 

INSTRUCTIONAL MATERIALS FOR READING 

Basal reading programs are a traditional part of reading instruction in this country. They 
typically include a compilation of reading passages and exercises, as well as ancillary 
materials, such as workbooks and tests. These types of programs account for at least 
two-thirds of all expenditures for reading instruction and are used in more than 95 percent 
of all school districts through grade 6. 29 However, other types of reading programs may 
utilize trade books, such as story or informational books, that are not necessarily published 
for the sole purpose of reading instruction. When students encounter a variety of texts, 
they expand their general understanding of language, as well as their understanding of text 
and its underlying structures. 30 To provide information about instructional materials used 
for fourth-grade classes, students’ reading teachers were asked to report about the type of 
materials that formed the core of their reading program. Table 13 and Table A13 
(Page 142) in the Data Appendix provide the results. According to Arkansas reading 
teachers: 



27 M.A. Foertsch. Reading In and Out of School . (Washington, DC: National Center for Education Statistics, 
1992). 

28 J.A. Dole, G.G. Duffy, L.R. Roehler, and P.D. Pearson. “Moving From the Old to the New: Research on 
Reading Comprehension Instruction,” Review of Educational Research, 61. (1991). 

29 Jeanne S. Chall and James R. Squire. “The Publishing Industry and Textbooks,” in R. Barr, M. Kamil, 

P. Mosenthal, and P.D. Pearson, Eds., Handbook of Reading Research, Volume II. (New York, NY: 
Longman, 1991). 

30 A. Applebee, J. Langer, and I. Mullis. Who Reads Best? (Princeton, NJ: National Assessment of 
Educational Progress, Educational Testing Service, 1988). 
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• Less than half of the fourth-grade public-school students in Arkansas 
(35 percent) had reading teachers who used both basal and trade books, 
more than half (63 percent) had reading teachers who primarily used basal 
readers, and none (0 percent) had reading teachers who primarily used 
trade books. 

• Less than half of the students attending schools in areas classified as 
“other” (35 percent), less than half of the students in disadvantaged urban 
areas (35 percent), and less than half of the students in extreme rural areas 
(35 percent) in Arkansas had reading teachers who used both basal and 
trade books. 

• More than half of the students attending schools in areas classified as 
“other” (63 percent), more than half of the students in disadvantaged 
urban areas (61 percent), and more than half of the students in extreme 
rural areas (63 percent) in Arkansas had reading teachers who primarily 
used basal readers. 

• None of the students attending schools in areas classified as “other” 
(0 percent), relatively few of the students in disadvantaged urban areas 
(2 percent), and none of the students in extreme rural areas (0 percent) in 
Arkansas had reading teachers who primarily used trade books. 

• Students in Arkansas wlpse teachers used both basal and trade books had 
about the same average reading proficiency as those whose teachers 
primarily used basal readers. 
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The NAEP reading scale ranges from 0 to 500. The standard errors of the statistics appear in parentheses. It 
can be said with about 95 percent confidence that, for each population of interest, the value for the entire 
population is within db 2 standard errors of the estimate for the sample. In comparing two estimates, one must 
use the standard error of the difference (see Appendix A for details). ! Interpret with caution - the nature of 
the sample does not allow accurate determination of the variability of this statistic. *** Sample size is 
insufficient to permit a reliable estimate (fewer than 62 students). 
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INSTRUCTIONAL RESOURCES FOR READING 



Teachers’ use of resources is obviously constrained by the availability of those resources. 
Thus, the assessed students* teachers were asked about the extent to which they were able 
to obtain all of the instructional materials and other resources they needed. From 
Table 14 and Table A14 (Page 144) in the Data Appendix: 

• In Arkansas, 10 percent of the fourth-grade students had reading teachers 
who reported getting all of the resources they needed, while 43 percent of 
the students were taught by teachers who got only some or none of the 
resources they needed. Across the nation, these figures were 11 percent 
and 39 percent, respectively. 

• In Arkansas, 8 percent of the students attending schools in areas classified 
as “other”, 6 percent of the students in disadvantaged urban areas, and 
1 1 percent of the students in extreme rural areas had reading teachers who 
got all of the resources they needed. 

• By comparison, 44 percent of the students attending schools in areas 
classified as “other”, 77 percent of the students in disadvantaged urban 
areas, and 36 percent of the students in extreme rural areas in Arkansas 
were in classrooms where only some or no resources were available. 

• Students in Arkansas whose teachers got all of the resources they needed 
had about the same average reading proficiency as those whose teachers got 
only some or none of the resources they needed. 
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Which of the following statements is true 
about how well your school system 
supplies you with the instructional 
materials and other resources you need to 
teach your class? 



I get all the resources I need. 

I get most of the resources I need. 

I get some or none of die resources I need. 









The NAEP reading scale ranges from 0 to 500. The standard errors of the statistics appear in parentheses. It 
can be said with about 95 percent confidence that, for each population of interest, the value for the entire 
population is within ± 2 standard errors of the estimate for the sample. In comparing two estimates, one must 
use the standard error of the difference (see Appendix A for details). ! Interpret with caution — the nature of 
the sample does not allow accurate determination of the variability of this statistic. 
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Teachers were also asked about their use of specific types of resources that can be used to 
add depth and variety to the reading program. As indicated in Table 15: 



• In Arkansas, 2 percent of the fourth-grade students had reading teachers 
who used children’s newspapers and/or magazines almost every day; 
11 percent of the students had reading teachers who used reading kits 
almost every day; 4 percent had reading teachers who used computer 
software for reading instruction almost every day; 23 percent had reading 
teachers who used a variety of books almost every day; and, finally, 
14 percent had teachers who used materials from other subject areas 
almost every day. 

• By comparison, in Arkansas, 47 percent of the fourth-grade students had 
reading teachers who never or hardly ever used children’s newspapers 
and/or magazines; 49 percent of the students had reading teachers who 
never or hardly ever used reading kits; 39 percent had reading teachers who 
never or hardly ever used computer software for reading instruction; 
18 percent had reading teachers who never or hardly ever used a variety 
of books; and 27 percent had teachers who never or hardly ever used 
materials from other subject areas. 
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How often do you use the following 
resources to teach reading? 

Children’s newspapers and/or magazines 

Almost every day 

Once or twice a week 
Once or twice a month 
Never or hardly ever 

Reading kits 

Almost every day 

Once or twice a week 
Once or twice a month 
Never or hardly ever 

Computer software for reading instruction 

Almost every day 

Once or twice a week 
Once or twice a month 
Never or hardly ever 

A variety of books (e g., novels, collections 
of poetry, nonfiction) 

Almost every day 

Once or twice a week 
Once or twice a month 
Never or hardly ever 

Materials from other subject areas 

Almost every day 

Once or twice a week 
Once or twice a month 
Never or hardly ever 



Proficiency 

2 ( 0 .?) 

: 23 ( 2.4) 

‘>'.215 ( 2 . 3 )>;: 
: 28 ($$$* 

210 ( 3 ; 3 ): 

47 ( 3 . 5 ) 

212 (2 2 ) 

11 ( 2 . 0 ) 

208 ( 3 . 6 ) 

17 ( 2 . 9 ) r 

...208 (| 3 . 6 )% 

23 ( 2 . 8 ) 

l 

49 ( 3» 

: y 213 ( 1 . 6 ) 

200 ( 8.0)! 

43 ( 4 . 8 ) 

. 209 ( 2 . 3 ) 

221 ( 3 . 8 ) 
39 ( 4 . 2 )'.. 

2|j|S9)* 



, ^ 



WSm&am 

24 ( 3.0) . 

>K21^(:2;9);i; 

18 ( 2.5) ! 

!<f 2^:[ZAY' ; 

■ 14 ( 2.2) > 

!(,: !’209 .(.‘4p3)r : y ; : 

’ 21 1 |(i2(5) . ' 

■ , :.3ll(c2^)'^ 

p 21.3M2.0) | 
B 27 ( 3.3) 



The NAEP reading scale ranges from 0 to 500. The standard errors of the statistics appear in parentheses. It 
can be said with about 95 percent confidence that, for each population of interest, the value for the entire 
population is within ± 2 standard errors of the estimate for the sample. In comparing two estimates, one must 
use the standard error of the difference (see Appendix A for details). ! Interpret with caution - the nature of 
the sample does not allow accurate determination of the variability of this statistic. *** Sample size is 
insufficient to permit a reliable estimate (fewer than 62 students). 
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EMPHASIS ON ASPECTS OF READING 



Expert readers use rapid decoding, large vocabularies, phonemic awareness, knowledge 
about text features, and a variety of strategies to aid comprehension and memory. 31 To 
examine the aspects of reading being emphasized in fourth-grade reading classrooms in 
Arkansas, public-school students’ reading teachers were asked to report on the amount of 
instructional time they devoted to five different aspects of reading: decoding skills, oral 
reading, vocabulary, comprehension/interpretation, and reading strategies. As shown in 
Table 16, according to their reading teachers: 



• In Arkansas, about one quarter of the fourth-grade students (24 percent) 
had reading teachers who devoted almost all of their instructional time in 
reading to teaching decoding skills; about one quarter of the students 
(25 percent) had reading teachers who devoted almost all of their 
instructional time in reading to oral reading; less than half (41 percent) had 
reading teachers who devoted almost all of their instructional time in 
reading to teaching vocabulary; more than half (57 percent) had reading 
teachers who devoted almost all of their instructional time in reading to 
comprehension/interpretation; and finally, less than half (31 percent) had 
teachers who devoted almost all of their instructional time in reading to 
reading strategies. 

• By comparison, in Arkansas, relatively few of the fourth-grade students 
(3 percent) had reading teachers who never or hardly ever devoted any 
instructional time to teaching decoding skills; relatively few of the students 
(3 percent) had reading teachers who never or hardly ever devoted 
instructional time to oral reading; relatively few (1 percent) had reading 
teachers who never or hardly ever devoted their instructional time to 
teaching vocabulary; relatively few (1 percent) had reading teachers who 
never or hardly ever devoted their time to comprehension/interpretation; 
and relatively few (3 percent) had teachers who never or hardly ever 
devoted their instructional time to reading strategies. 



31 L. Baker and A.L. Brown. “Metacognitive Skills and Reading,” in P.D. Pearson, M. Kamil, R. Barr, and 
P. Mosenthal, Eds., Handbook of Reading Research (Vol. I). (While Plains, NV: Longman, 1984).; 

R.C. Anderson, E.H. Hieberl, J.A. Scott, and I.A.G. Wilkinson. Becoming a Nation of Readers: The Report 
of the Commission on Reading . (U.S. Department of Education: The National Institute of Education, 1985).; 
J.A. Dole, G.G. Duffy, L.R. Roehler, and P.D. Pearson. “Moving From the Old to the New: Research on 
Reading Comprehension Instruction,” Review of Educational Research, 61. (1991). 

80 



74 



THE 1992 NAEP TRIAL STATE ASSESSMENT 



Arkansas 



THE NATION’S 
REPORT 
CARD 




Trial State Assessment 



table 16 | Teachers’ Reports on Emphasis on 

I Aspects of Reading 



Arkansas 



Southeast 



Nation 





How much of your instructional 




Rercentege 


P«ri^nta^|§| 






time in reading do you devote to 






and ^ S : jj|l 1 | 


i§(P§ 




each of the following? 






Proficilrt^^l| 




Decoding skills 
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Some of the time 
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Oral reading 




■ : 


- : if : ; : ’ i 
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3 ( 1.3} 












11^2261(1^181 | 


Vocabulary 
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Comprehension / Interpretation 
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The NAEP reading scale ranges from 0 to 500. The standard errors of the statistics appear in parentheses. It 
can be said with about 95 percent confidence that, for each population of interest, the value for the entire 
population is within ± 2 standard errors of the estimate for the sample. In comparing two estimates, one must 
use the standard error of the difference (see Appendix A for details). ! Interpret with caution - the nature of 
the sample does not allow accurate determination of the variability of this statistic. *** Sample size is 
insufficient to permit a reliable estimate (fewer than 62 students). 
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INSTRUCTIONAL ACTIVITIES 

Teachers can nurture students’ reading comprehension ability by providing instructional 
activities that prepare students for a wide variety of specific reading tasks. 32 These 
activities support students’ understanding of the text being read and model the ways in 
which students can control the process of building meaning when reading on their 
own. 33 To provide information about the instructional activities in which fourth-grade 
public-school students are engaged, the students participating in the Trial State Assessment 
Program and their reading teachers were asked to report on the frequency with which the 
teachers asked the students to do a variety of activities. The students’ and teachers’ 
responses are presented in the three following sections - workbooks, worksheets, and 
writing; discussions and group activities; and time to read. 

WORKBOOKS, WORKSHEETS, AND WRITING 

Children spend considerably more time completing workbook assignments than they do 
receiving instruction from their teachers. 34 However, analyses of workbook activities 
reveal that many of these activities require only a perfunctory level of reading. 3 s Few 
workbook activities require students to do any extended writing. However, opportunities 
to write have been found to contribute to knowledge of how written and oral language are 
related, and to growth in phonics, spelling, vocabulary development, and reading 
comprehension. 36 

To examine the use of workbooks, worksheets, and the reading/writing connection, 
students and their reading teachers were asked about the frequency with which teachers 
asked students to work in a reading workbook or on a worksheet, to write about something 
they had read, or to write in a log or journal about what they had read. Table 17 provides 
these results. 



32 S.G. Paris. "Teaching Children to Guide Their Reading and Learning,” in Taffy E. Raphael, Ed., The 
Contexts of School-Based Literacy. (New York, NY: Random House, 1984). pp. 115-130. 

33 M.A. Foertsch. Reading In and Out of School. (Washington, DC: National Center for Education Statistics, 
1992). 

34 R.C. Anderson, E.H. Hiebert, J.A. Scott, and I.A.G. Wilkinson. Becoming a Nation of Readers: The Report 
of the Commission on Reading. (U.S. Department of Education: The National Institute of Education, 1985). 

35 J. Osborn. "The Purposes, Uses, and Contents of Workbooks and Some Guidelines for Publishers,” in 
R.C. Anderson, J. Osborn, and R.J. Tierney (Eds.), Learning to Read in American Schools: Basal Readers 
and Content Texts. (Hillsdale, NJ: Erlbaum, 1984). J. Osborn. "Workbooks: Counting, Matching, and 
Judging,” in J. Osborn, P.T. Wilson, and R.C. Anderson (Eds.), Reading Education: Foundations for a 
Literate America. (Lexington, MA: Lexington Books, 1985). 

36 R.J. Tierney and M. Leys. "What is the Value of Connecting Reading and Writing?” in B. Peterson, Ed., 
Convergences: Essays on Reading, Writing, and Literacy. (Urbana, IL: National Council of Teachers of 
English, 1986). 
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According to the fourth -grade students: 

• More than half of the fourth-grade students in Arkansas (65 percent) were 
asked to work in a reading workbook or on a worksheet almost every day 
while some (14 percent) were asked to work in a reading workbook or on 
a worksheet less than weekly. 

• In Arkansas, 19 percent of the students were asked to write about 
something they have read almost every day; 48 percent were asked to do 
this less than weekly. 

• Some of the students in Arkansas (14 percent) were asked to write in a log 
or journal about what they have read almost every day; more than half 
(66 percent) were given time to do this activity less than weekly. 



And, according to their reading teachers: 



• About half of the fourth-grade students in Arkansas (45 percent) were 
asked to work in a reading workbook or on a worksheet almost every day 
while relatively few (2 percent) were asked to do these activities less than 
weekly. 

• In Arkansas, 6 percent of the students were asked to write about something 
they have read almost every day; 40 percent were asked to write about 
something they have read less than weekly. 

• Some of the students in Arkansas (11 percent) were asked to write in a log 
or journal about what they have read almost every day; about three 
quarters (71 percent) were given time to write in a log or journal less than 
weekly. 
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TABLE 17 



Teachers’ and Students’ Reports on 
Workbooks, Worksheets, and 
Writing 



Arkansas 


Southeast 


Nation 


Teacher 


Student 


Teacher 


Student 


Teacher 


Student 



How often do you (does 
your teacher) do each 
of the following as a 
part of reading 
instruction? 



Ask students to work in a 
reading workbook or on a 
worksheet 

Almost every day 

At least once a week 

Less than weekly 

Ask students to write about 
something they have read 

Almost every day 

At least once a week 

Less than weekly 

Ask students to write in a 
log or journal about what they 
have read 



- v. "A x ■■ . '-V.\ .. . I'x 

; Percentage * 
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■ : : '■ . 
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||||3.8) ' ' 65 ( 1.5) 
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•«» (**.*) 
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220 ( 4.1)! 

; 54 ( 3.3! 
211 ( 1.7) 

40(3.5). 
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Almost every day 
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218 ( 5.0) 209 ( 3.8): 


At least once a week 


19 ( 2.9} 1 20 ( 0.9) 


;213 ( 3.9) 2ip|ggfH 


Less than weekly 


71 ( 3.2) 66 ( 1.3) 

211(1.5) 218(1.3) 


g<212 (^g216T»2|3)| 



Percentage 
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; -S- ..... 






WM3M 
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49 ( 5.4) 
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17 ( 6.0) 
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49 ( 5.8) 
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217 ( 1.9) 

2® ( 2.5) 
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210 ( 1.3) 

fmmp 
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1 
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The NAEP reading scale ranges from 0 to 500. The standard errors of the statistics appear in parentheses. It 
can be said with about 95 percent confidence that, for each population of interest, the value for the entire 
population is within ± 2 standard errors of the estimate for the sample. In comparing two estimates, one must 
use the standard error of the difference (see Appendix A for details). ! Interpret with caution - the nature of 
the sample does not allow accurate determination of the variability of this statistic. *** Sample size is 
insufficient to permit a reliable estimate (fewer than 62 students). 
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DISCUSSION AND GROUP ACTIVITIES 



Discussion-related activities are an important part of classroom learning, because they 
provide opportunities for students to ask questions about things they do not understand 
or want to know more about. A lack of emphasis on group work or the sharing of different 
interpretations limits opportunities students have for discovering that their reactions or 
interpretations may not be the only ones justified by the text. 37 

To examine the prevalence of discussion-related activities, students and their reading 
teachers were asked about how frequently the students were asked to discuss new or 
difficult vocabulary, to talk with each other about what they have read, or to do a group 
activity or project about what they have read. As shown in Table 18: 



According to the fourth-grade students: 

• Less than half of the fourth-grade students in Arkansas (35 percent) were 
asked to discuss new or difficult vocabulary almost every day while about 
one quarter (29 percent) were asked to do this activity less than weekly. 

• In Arkansas, 16 percent of the students were asked to talk with each other 
about what they have read almost every day; 59 percent were asked to do 
this less than weekly. 

• Relatively few of the students in Arkansas (10 percent) were asked to do 
a group activity or project about what they have read almost every day; 
about three quarters (70 percent) were given time to do this activity less 
than weekly. 



And, according to their reading teachers: 



• Less than half of the fourth-grade students in Arkansas (43 percent) were 
asked to discuss new or difficult vocabulary almost every day while 
relatively few (1 percent) were asked to do this activity less than weekly. 

• In Arkansas, 19 percent of the students were asked to talk with each other 
about what they have read almost every day; 35 percent were asked to do 
this less than weekly. 

• Relatively few of the students in Arkansas (2 percent) were asked to do a 
group activity or project about what they have read almost every day; many 
(87 percent) were given time to do this activity less than weekly. 



BEST COPV AVAILABLE 



37 J. Moffett and B. Wagner. “Student Centered Reading Activities,” English Journal, 80. 1991. 
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TABLE 18 



Teachers’ and Students’ Reports on 
the Frequency of Discussion and 
Group Activities 



Arkansas 


Southeast 


Nation 


Teacher 


Student 


Teacher 


Student 


Teacher 


Student 



How often do you (does 
your teacher) do each 
of the following as a 
part of reading 
instruction? 



Discuss new or difficult 
vocabulary 

Almost every day 



At least once a week 



Less than weekly 



Ask students to talk to each 
other about what they have read 

Almost every day 



At least once a week 



Less than weekly 



Ask students to do a group 
activity or project about what 
they have read 

Almost every day 



At least once a week 



Less than weekly 
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The NAEP reading scale ranges from 0 to 500. The standard errors of the statistics appear in parentheses. It 
can be said with about 95 percent confidence that, for each population of interest, the value for the entire 
population is within ± 2 standard errors of the estimate for the sample. In comparing two estimates, one must 
use the standard error of the difference (see Appendix A for details). ! Interpret with caution - the nature of 
the sample does not allow accurate determination of the variability of this statistic. *** Sample size is 
insufficient to permit a reliable estimate (fewer than 62 students). 
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TIME TO READ 



Independent reading is probably a major source of reading fluency. In contrast to 
completing workbook pages or computer drills, the reading of books provides practice in 
the whole act of reading. However, the amount of time children spend reading in the 
average classroom, as well as the number of pages read for school and homework, has been 
found to be negligible. 38 

Both the fourth-grade students and their reading teachers were questioned about the 
frequency with which the teachers asked the students to read aloud or read silently, or gave 
the students time to read books of their own choosing. Table 19 provides this information. 



According to the fourth-grade students: 

• In Arkansas, 49 percent of the students were asked to read aloud almost 
every day, while 23 percent were asked to read aloud less than weekly. 

• More than half of the fourth-grade students in Arkansas (58 percent) were 
asked to read silently almost every day; some (14 percent) were asked to 
read silently less than weekly. 

• About half of the students in Arkansas (50 percent) were given time to read 
books of their own choosing almost every day; some (20 percent) were 
given time to read books they had chosen less than weekly. 



And, according to their reading teachers: 

• In Arkansas, 40 percent of the students were asked to read aloud almost 
every day, while 5 percent were asked to read aloud less than weekly. 

• About half of the fourth-grade students in Arkansas (49 percent) were 
asked to read silently almost every day; relatively few (6 percent) were 
asked to read silently less than weekly. 

• About half of the students in Arkansas (54 percent) were given time to read 
books of their own choosing almost every day; some (16 percent) were 
given time to read books they had chosen less than weekly. 



38 R.C. Anderson, E.H. Hiebert, J.A. Scott, and I.A.G. Wilkinson. Becoming a Nation of Readers: The Report 
of the Commission on Reading. (U.S. Department of Education: The National Institute of Education, 1985). 
M.A. Foertsch. Reading In and Out of School. (Washington, DC: National Center for Education Statistics, 
1992). 
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table 19 | Teachers’ and Students’ Reports on 

I the Frequency of Reading in Class 



Arkansas 


Southeast 


Nation 


Teacher 


Student 


Teacher 


Student 


Teacher 


Student 



How often do you (does 
your teacher) do each 
of the following as a 
part of reading 
instruction? 



Ask students to read aloud 

Almost every day 

At least once a week 
Less than weekly 

Ask students to read silently 

Almost every day 

At least once a week 

Less than weekly 

Give students time to read books 
they have chosen for themselves 

Almost every day 

At least once a week 
Less than weekly 
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The NAEP reading scale ranges from 0 to 500. The standard errors of the statistics appear in parentheses. It 
can be said with about 95 percent confidence that, for each population of interest, the value for the entire 
population is within ± 2 standard errors of the estimate for the sample. In comparing two estimates, one must 
use the standard error of the difference (see Appendix A for details). ! Interpret with caution — the nature of 
the sample does not allow accurate determination of the variability of this statistic. *** Sample size is 
insufficient to permit a reliable estimate (fewer than 62 students). 



READING AND USE OF LIBRARIES 

Analysis of schools that have been successful in promoting independent reading suggest 
that one of the keys is ready access to books. 39 Libraries can be a major resource in 
developing students’ reading abilities because students can use them as quiet places to read 
as well as to check out books and to obtain reference information. Thus, to examine 
library use, students’ reading teachers were asked about the frequency with which they sent 
or took their reading classes to the library and assigned students to read a book from the 
library. 



39 R.C. Anderson, E.H. Hiebert, J.A. Scott, and I.A.G. Wilkinson. Becoming a Nation of Readers: The Report 
of the Commission on Reading. (U.S. Department of Education: The National Institute of Education, 1985). 
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Table 20 and Table A20 (Page 180) in the Data Appendix provide the results from teachers’ 
reports about the frequency of sending fourth-grade students to the library: 



• Many of the students in Arkansas (88 percent) had reading teachers who 
sent or took the class to the library at least once a week; relatively few 
(9 percent) had reading teachers who never or hardly ever sent or took the 
class to the library. 

• In Arkansas, 90 percent of the students attending schools in areas classified 
as “other”, 89 percent of the students in disadvantaged urban areas, and 
8 1 percent of the students in extreme rural areas had reading teachers who 
sent or took the class to the library at least once a week. 

• By contrast, 7 percent of the students attending schools in areas classified 
as “other”, 11 percent of the students in disadvantaged urban areas, and 
15 percent of the students in extreme rural areas in Arkansas had reading 
teachers who never or hardly ever sent or took the class to the library. 
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table 20 | Teachers’ Reports on Sending 

I Students to the Library 



Arkansas 



Southeast 



Nation 









Percentage 
| Proficiency 




How often do you send or take 
the class to the library? 




Percentage 

ilgiml 

" Pi^ficiericy 


Percentage 

and 

Proficiency 


At least once a week 


88 ( 2.8) 
213 ( 1.4) . 


83 ( 5.7) 
214 ( 3.7) 


'85 ( 2.7) . 
.. 219(1.5) 


Once or twice a month 


lH 3 M.5) | ; 

206 ( 7.0)1 


14 ( 5.0) 


V 9 ( 1.9) 

' 208 ( 4.2)! 


Never or hardly ever 


9(2.3) 
207 ( 3.9)! 


. 2 ( 1.0) 


208 ( 4.4)! 



The NAEP reading scale ranges from 0 to 500. The standard errors of the statistics appear in parentheses. It 
can be said with about 95 percent confidence that, for each population of interest, the value for the entire 
population is within ± 2 standard errors of the estimate for the sample. In comparing two estimates, one must 
use the standard error of the difference (see Appendix A for details). Percentages may not add to 100 because 
a very small percentage of teachers reported that there was no library at their school. ! Interpret with caution 
- the nature of the sample does not allow accurate determination of the variability of this statistic. *** Sample 
size is insufficient to permit a reliable estimate (fewer than 62 students). 
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Table 21 and Table A21 (Page 182) in the Data Appendix provide results about teachers’ 
reports on the frequency of assigning students to read a book from the library: 

• About half of the fourth graders in Arkansas (45 percent) had reading 
teachers who assigned reading a book from the library at least once a week; 
about one quarter (24 percent) had reading teachers who never or hardly 
ever assigned reading library books. 

• About half of the students attending schools in areas classified as “other” 

(48 percent), more than half of the students in disadvantaged urban areas 
(61 percent), and less than half of the students in extreme rural areas 
(35 percent) in Arkansas had reading teachers who assigned students to 
read a book from the library at least once a week. 

• By comparison, some of the students attending schools in areas classified 
as “other” (20 percent), relatively few of the students in disadvantaged 
urban areas (7 percent), and less than half of the students in extreme rural 
areas (39 percent) in Arkansas had reading teachers who never or hardly 
ever assigned students to read a book from the library. 
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table 21 | Teachers’ Reports on Assigning 

I Books from the Library 



Arkansas 



Southeast 



Nation 



How often do you assign 
students to read a book from 
the library? 



At least once a week 



Once or twice a month 



Never or hardly ever 




. and 



.Proflclenc/ 






31 ( 3 . 9 ) 
214 ( 2 . 1 ) 

24 ( 3 . 5 ) 






Percen tage 






gjffSfi h •' ’.V IV 




IV V A o ' { I ST 4 \Y 




• ■ -• ■ : : -x-V - • ' 







The NAEP reading scale ranges from 0 to 500. The standard errors of the statistics appear in parentheses. It 
can be said with about 95 percent confidence that, for each population of interest, the value for the entire 
population is within ± 2 standard errors of the estimate for the sample. In comparing two estimates, one must 
use the standard error of the difference (see Appendix A for details). Percentages may not add to 100 because 
a very small percentage of teachers reported that there was no library at their school. ! Interpret with caution 
— the nature of the sample does not allow accurate determination of the variability of this statistic. 
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ASSESSING PROGRESS IN READING 



According to Becoming a Nation of Readers , standardized tests do not provide a deep 
assessment of reading comprehension and should be supplemented with observations of 
reading fluency, critical analysis of lengthy reading selections, and measures of the amount 
of independent reading and writing done by children 40 



Fourth-grade students’ reading teachers were asked a series of individual questions to report 
on how often they used different types of assessment measures — including multiple-choice 
tests, longer extended constructed-response questions, and reading portfolios — to assess 
student progress in reading. The use of reading portfolios is a relatively new practice and 
may not be widely used in many schools as an assessment tool. From Table 22: 

• Less than half of the fourth-grade students in Arkansas (33 percent) were 
assessed with multiple-choice tests once or twice a week while relatively few 
(5 percent) were never or hardly ever assessed in this manner. 

• In Arkansas, 27 percent of the students were asked to write paragraphs 
about what they had read once or twice a week; 14 percent were never or 
hardly ever assessed using these extended constructed-response questions. 

• Relatively few of the students in Arkansas (7 percent) were assessed by 
using reading portfolios about once or twice a week; more than half 
(59 percent) were never or hardly ever asked to do this activity. 



40 R.C. Anderson, E.H. Hiebert, J.A. ScoU, and l.A.G. Wilkinson. 
of the Commission on Reading. (U.S. Department of Education: 



Becoming a Nation of Readers: The Report 
The National Institute of Education, 1985). 
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TABLE 22 



Teachers’ Reports on Assessing 
Progress in Reading 



Arkansas 



Southeast 



Nation 



How often do you use each of the 
following to assess student progress in 
reading? 

Multiple-choice tests 

Once or twice a week 

Once or twice a month 
Once or twice a year 
Never or hardly ever 

Writing paragraphs about what they have read 

Once or twice a week 

Once or twice a month 
Once or twice a year 
Never or hardly ever 

Reading portfolios 

Once or twice a week 

Once or twice a month 
Once or twice a year 
Never or hardly ever 



m 



Percentage! 



Proficiency 






w 50 ( 3.5) 

13 ( 2.5! 
214(3.2)1 

5(1.3) 

211(5.2)! 

V ; 212 (2.0) ; 

VMM ( 3.2) . 

: : 213 ( 1:9 jtf); 

12 ( 2 . 0 ) 



III 7( 2.2) . 

205 6.1)! 

j pBM | 

; 210 ( 2.9) 

19 ( 2.7) 

214 ( 2.5) 

V 59(3.8) " 

;.7 ::; : 212 (1,7) 



■ ■ '■ : 



Percentage 
\ am* ■ 
Proficiency 

• : ' 22 ( 4.2) | 

205 ( 4.3)1 

58 ( $, 2 ) 

218 ( 2.0) 

■ e'1'2.8). 

209 ( 34)! 

11 (4.8) 

220 (15,7)1 

38(4.1) 

216(4.2) 

47 ( 4.7) 

mm 

207 ( 5.6)1 
5 ( 1.2) 
M M‘V) 

. 11 (3.1) 
217(12.9)! 

. 23 ( 4.8) 

219(5,7)! 

14 ( 3.5) 

215 ( 6.5)1 

■ 52 ( 8.0) 

209 ( 1.5) 



■iliiplliii 

iiiiliiiii 

49 (3.3) 

218 ( 1.7) 

221 ( 2.5) 

21 ( 3.4) 
219(3,5) 

fill; 



Percentage 

ISS^ilSi; 

Proficiency 

mmm tiM SSi 



I ! 39 ( 2.6) II 
218 ( 1.6) 

8(1.4) 

212(3.9) 

6 ( 1.3) 

207 ( 4.5)1 

14 ( 1.8) . 

I 218(4.3) 
25(2.3) 

222 ( 2.4) 

13(2.3) 

47(3.3) 

Mksmmwmm 



The NAEP reading scale ranges from 0 to 500. The standard errors of the statistics appear in parentheses. It 
can be said with about 95 percent confidence that, for each population of interest, the value for the entire 
population is within ± 2 standard errors of the estimate for the sample. In comparing two estimates, one must 
use the standard error of the difference (see Appendix A for details). ! Interpret with caution - the nature of 
the sample does not allow accurate determination of the variability of this statistic. *** Sample size is 
insufficient to permit a reliable estimate (fewer than 62 students). 
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SUMMARY 



For instruction to be most effective, subject matter, teaching materials and activities, and 
the instructional context must be carefully orchestrated to create a meaningful and 
motivating learning experience. 41 Because classroom instructional time is typically limited, 
teachers need to make the best possible use of what is known about effective instructional 
delivery practices and resources. 



• In Arkansas, 35 percent of the fourth-grade public-school students had 
reading teachers who used both basal and trade books, 63 percent had 
reading teachers who primarily used basal readers, and 0 percent had 
reading teachers who primarily used trade books. 

• In Arkansas, 10 percent of the fourth-grade students had reading teachers 
who reported getting all of the resources they needed, while 43 percent of 
the students were taught by teachers who got only some or none of the 
resources they needed. Across the nation, these figures were 11 percent 
and 39 percent, respectively. 

• In Arkansas, 2 percent of the fourth-grade students had reading teachers 
who used children’s newspapers and/or magazines almost every day; 
11 percent of the students had reading teachers who used reading kits 
almost every day; 4 percent had reading teachers who used computer 
software for reading instruction almost every day; 23 percent had reading 
teachers who used a variety of books almost every day; and, finally, 
14 percent had teachers who used materials from other subject areas 
almost every day. 

• In Arkansas, 24 percent of the fourth-grade students had reading teachers 
who devoted almost all of their instructional time in reading to teaching 
decoding skills; 25 percent of the students had reading teachers who 
devoted almost all of their instructional time in reading to oral reading; 
41 percent had reading teachers who devoted almost all of their 
instructional time in reading to teaching vocabulary; 57 percent had 
reading teachers who devoted almost all of their instructional time in 
reading to comprehension/interpretation; and finally, 31 percent had 
teachers who devoted almost all of their instructional time in reading to 
reading strategies. 

• Many of the students in Arkansas (88 percent) had reading teachers who 
sent or took the class to the library at least once a week; relatively few 
(9 percent) had reading teachers who never or hardly ever sent or took the 
class to the library. 

• In Arkansas, 45 percent of the students had reading teachers who assigned 
reading a book from the library at least once a week; 24 percent had 
reading teachers who never or hardly ever assigned reading library books. 

• In Arkansas, 27 percent of the students were asked to write paragraphs 
about what they had read once or twice a week; 14 percent were never or 
hardly ever assessed using these extended constructed-response questions. 



41 J.I. Goodlad. A Place Called School: Prospects Jor the Future. (New York, NY: McGraw-Hill, 1984). 
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CHAPTER 5 

Who Is Teaching Reading to Fourth Graders? 

PREPARATION AND EXPERIENCE 



Many states have begun to raise teacher certification standards and strengthen teacher 
training programs. In curriculum areas requiring special attention and improvement, such 
as reading, it is particularly important to have well-qualified teachers. To provide 
information about the staff who are teaching reading to fourth-grade students in public 
schools, the Trial State Assessment gathered details on the teachers’ educational 
backgrounds. 

Table 23 summarizes teacher responses to questions concerning their academic preparation, 
certification, and their years of elementary or secondary teaching experience: 



• In Arkansas, 33 percent of the students were being taught by reading 
teachers who reported having at least a master’s or education specialist’s 
degree. This compares with 46 percent for students across the nation. 

• About half of the students (52 percent) had reading teachers who had the 
highest level of teaching certification that is recognized by Arkansas. This 
is about the same as the figure for the nation, where more than half of the 
students (57 percent) were taught by reading teachers who were certified 
at the highest level available in their states. 

• In Arkansas, 14 percent of the fourth-grade public-school students were 
being taught reading by teachers who had an undergraduate major in 
English, reading, and/or language arts. By comparison, 22 percent of the 
students across the nation had reading teachers with the same major. 

• Some of the fourth-grade public-school students in Arkansas (12 percent) 
were taught reading by teachers who had a graduate major in English, 
reading, and/or language arts. Across the nation, some (18 percent) of the 
students were taught by teachers who majored in English, reading, and/or 
language arts in graduate school. 
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0 In Arkansas, 65 percent of the fourth-grade public- school students were 
being taught reading by teachers who have taught at either the elementary 
or secondary level for at least 11 years (including part-time teaching). 
Across the nation, 69 percent of the students had reading teachers with at 
least 1 1 years’ experience. 
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TABLE 23 



Teachers’ Reports on Their Fields 
of Study and Teaching Experience 



Arkansas 



Southeast 



Nation 





What is the highest academic degree you hold? 




i Percentage)) 1 


PercentagfJJ 


Percentage 


Bachelor’s degree 


67 ( 3.2) 




54 ( 2.7) I 


Master’s or specialist’s degree 






; 45 ( mm 


Doctorate or professional degree 




H i ( iSiaf 






What type of teaching certification do you have 












that is recognized by Arkansas? 










None, temporary, probational, provisional, or emergency 


0 ( 0.4) 


lii 


7(1.2) 


Regular certification but less than the highest available 






37 ( 2.7) 


Highest certification available 






v 




What was your undergraduate major? 










English, reading, and/or language arts 


iiiiiisfll 




;L;22 fas); % 


Education 


S.82 SSiHf 




69 ( 3.5) 


Other 


: 4(1.0) 


A12 { 5J5) » s 


I- ifiggii 




What was your graduate major? 




• ■ 


' V > v ' 'o & 


1 ' : ' 


English, reading, and/or language arts 




H|l5 ( 4.5) 




Education 


51 ( 3.6) 


WssSI&m 


ipSIlSjllS 


Other or no graduate-level study 


' 36 ( 3i5 )yy 


{ '32;( 5.8) 


26 (§jjggig 




How many years in total have you taught at 




3 ...... 


-) ■: : ) )■ 






either the elementary or secondary level? 




. 






2 years or less 






7 ( 1.4) 


3-5 years 




17 ( 3.1) 


13 ( 1.9) 


6-10 years 


A; -17 ( 2.4)- 


■ .12X2:6).:. 


12 ! 1.4) 


1 1 -24 years 


50 ( 3.5; 


50 ( 5.1) 


S&5lSftf£)/ M- 


25 years or more 


16 ( 2.4) 


15 (3.2) 





The standard errors of the statistics appear in parentheses. It can be said with about 95 percent confidence that, 
for each population of interest, the value for the entire population is within ± 2 standard errors of the estimate 
for the sample. In comparing two estimates, one must use the standard error of the difference (see 
Appendix A for details). 
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PROFESSIONAL DEVELOPMENT 



Teachers also were asked about the amount of time they spent on in-service education 
dedicated to reading or the teaching of reading during the year immediately preceding the 
Trial State Assessment (Table 24): 



• In Arkansas, 33 percent of the fourth-grade public-school students had 
reading teachers who spent at least 16 hours on in-service education 
dedicated to reading or the teaching of reading. Across the nation, 
31 percent of the students had reading teachers who spent at least that 
much time on similar types of in-service education. 

• In Arkansas, 10 percent of the students had reading teachers who spent no 
time on in-service education devoted to reading or the teaching of reading. 
Nationally, 9 percent of the students had reading teachers who spent no 
time on similar in-service education. 
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TABLE 24 



Teachers’ Reports on Their 
In-Service Education 



Arkansas 



Southeast 



Nation 




The standard errors of the statistics appear in parentheses. It can be said with about 95 percent confidence that, 
for each population of interest, the value for the entire population is within ± 2 standard errors of the estimate 
for the sample. In comparing two estimates, one must use the standard error of the difference (see 
Appendix A for details). 
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Finally, teachers were asked to report on whether they had training in specific aspects of 
reading during the past five years, either in college courses or through in-service education. 
As indicated in Table 25: 



• In Arkansas, 82 percent of the fourth-grade public-school students had 
reading teachers who reported that they had training in teaching critical 
thinking skills; 83 percent had reading teachers who reported having 
training in combining reading and writing; 68 percent had reading teachers 
who reported having training in the whole language approach to teaching 
reading; and 72 percent had reading teachers who reported having training 
in reading assessment. 

• Across the nation, 83 percent of the fourth-grade public-school students 
had reading teachers who reported that they had training in teaching critical 
thinking skills; 89 percent had reading teachers who reported having 
training in combining reading and writing; 80 percent had reading teachers 
who reported having training in the whole language approach to teaching 
reading; and 75 percent had reading teachers who reported having training 
in reading assessment. 



THE NATION S 
REPORT 
CARD 



1992 






Trial State Assessment 



TABLE 25 



Teachers’ Reports on Training in 
Specific Reading Areas 



Arkansas 



Southeast 



Nation 













Percentage of students whose teachers 
have had training in each of the following 
areas during the past five years 




Percentage 


Percentage 


Percentage 


Teaching critical thinking skills 


i 

82 ( 2.7) ; 


88 ( 2.4) 


83 ( 1.8) 


Combining reading and writing 


83 ( 2.3) 


92 ( 1.8) 


80 ( 1.7) 


The whole language approach to 
teaching reading 


68 ( 3.4) 


78 ( 2.3) 


80(2.0) 


Reading assessment 


72 ( 3.4) 


70(3.9) 


j| 



The standard errors of the statistics appear in parentheses. It can be said with about 95 percent confidence that, 
for each population of interest, the value for the entire population is within ± 2 standard errors of the estimate 
for the sample. In comparing two estimates, one must use the standard error of the difference (see 
Appendix A for details). 
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SUMMARY 



In recent years, accountability for educational outcomes has become an issue of increasing 
importance to federal, state, and local governments. There is no guarantee that individuals 
with a specific set of credentials will be effective teachers; however, it is likely that relevant 
training and experience do contribute to better teaching. 



The information about teachers’ educational backgrounds and experience reveals that: 



• In Arkansas, 33 percent of the students were being taught by reading 
teachers who reported having at least a master’s or education specialist’s 
degree. This compares with 46 percent for students across the nation. 

• About half of the students (52 percent) had reading teachers who had the 
highest level of teaching certification that is recognized by Arkansas. This 
is about the same as the figure for the nation, where more than half of the 
students (57 percent) were taught by reading teachers who were certified 
at the highest level available in their states. 

• In Arkansas, 14 percent of the fourth-grade public-school students were 
being taught reading by teachers who had an undergraduate major in 
English, reading, and/or language arts. By comparison, 22 percent of the 
students across the nation had reading teachers with the same major. 

• Some of the fourth-grade public-school students in Arkansas (12 percent) 
were taught reading by teachers who had a graduate major in English, 
reading, and/or language arts. Across the nation, some (18 percent) of the 
students were taught by teachers who majored in English, reading, and/or 
language arts in graduate school. 

• In Arkansas, 65 percent of the fourth-grade public-school students were 
being taught reading by teachers who have taught at either the elementary 
or secondary level for at least 11 years (including part-time teaching). 
Across the nation, 69 percent of the students had reading teachers with at 
least 1 1 years’ experience. 

• In Arkansas, 33 percent of the fourth-grade public-school students had 
teachers who spent at least 16 hours on in-service education dedicated to 
reading or the teaching of reading. Across the nation, 31 percent of the 
students had teachers who spent at least that much time on similar types 
of in-service education. By comparison, in Arkansas, 10 percent of the 
students had reading teachers who spent no time on in-service education 
devoted to reading or the teaching of reading. Nationally, 9 percent of the 
students had reading teachers who spent no time on similar in-service 
education. 

• In Arkansas, 82 percent of the fourth-grade public-school students had 
reading teachers who reported that they had training in teaching critical 
thinking skills; 83 percent had reading teachers who reported having 
training in combining reading and writing; 68 percent had reading teachers 
who reported having training in the whole language approach to teaching 
reading; and 72 percent had reading teachers who reported having training 
in reading assessment. 
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CHAPTER 6 



Students’ Home Support for Literacy 



Home and attitudinal variables affect students 1 reading achievement. 42 In addition, good 
readers usually interact with a wide variety of materials on their own, and share their 
experiences with family and friends. 43 Thus, it is important to understand students’ 



of home support that is available for reading. To examine these factors, students 
participating in the Trial State Assessment were asked a series of questions about 
themselves, their parents or guardians, and home factors related to reading. 

READING OUTSIDE OF SCHOOL 

Because relatively small percentages of students appear to devote little or no time to leisure 
reading, 44 students participating in the Trial State Assessment were asked to report on how 
often they read for fun on their own time (Table 26). They also were asked about the 
number of books they have read on their own outside of school during the month 
preceding the assessment (Table 27), and how often they have taken books out of the 
school library or public library for their own enjoyment (Table 28). 



42 J.T. Guthrie and V. Greaney. “Literacy Acts,” in R. Barr, M. Kamil, P. Mosenthal, and P.D. Pearson, Eds., 
Handbook of Reading Research: Volume II. (New York, NY: Longman, 1991). 

43 M.A. Foertsch. Reading In and Out of School. (Washington, DC: National Center for Education Statistics, 
1992). 

44 L.G. Fielding, P.T. Wilson, R.C. Anderson. “A New Focus on Free Reading: The Role of Trade Books in 
Reading and Instruction,” in T. Raphael and R. Reynolds, Eds., Contexts of Literacy. (New York: Longman, 
1990); V. Greandy. “Factors Related to Amount and Type of Leisure-time Reading,” Reading Research 
Quarterly, 15(80). (1980). pp. 337-357. 
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The results are provided in Tables 26 and A26 (Page 190) regarding how often students 
reported reading for fun on their own time. 

• In Arkansas, 39 percent of the fourth-grade public- school students 
reported that they read for fun almost every day while 16 percent never or 
hardly ever did so. 

• In Arkansas, a smaller percentage of boys than girls read for fun almost 
every day; a greater percentage of boys than girls never or hardly ever did. 

• Less than half of the White students (39 percent), less than half of the 
Black students (38 percent), less than half of the Hispanic students 
(38 percent), and less than half of the American Indian students 
(38 percent) in Arkansas read for fun almost every day. 

• Some of the White students (15 percent), some of the Black students 
(18 percent), some of the Hispanic students (20 percent), and some of the 
American Indian students (14 percent) in Arkansas never or hardly ever 
read for fun. 
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TABLE 26 



Students’ Reports on Reading for 
Fun 



Arkansas 



Southeast 



Nation 



How often do you read for fun 
on your own time? 

Almost every day 
Once or twice a week 
Once or twice a month 
Never or hardly ever 



Percentage 

Proficferi^^te 

§ 39 ( 1.2) 1 

217(1.9) ; 

34 (1.0) 
.213(1.3) 

12 



•ift/ng)' ,A l: 



Percentage 



Pro|||^p^p 
40(1.8) 













\ 201 ( 3.4) 



Percentage 

* ' Proflt8«i«y|S®t 







4 1 ‘ O \ ,,<v ' 









The NAEP reading scale ranges from 0 to 500. The standard errors of the statistics appear in parentheses. It 
can be said with about 95 percent confidence that, for each population of interest, the value for the entire 
population is within ± 2 standard errors of the estimate for the sample. In comparing two estimates, one must 
use the standard error of the difference (see Appendix A for details). 
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Concerning how many books were read by fourth-grade students in Arkansas, Table 27 
and Table A27 (Page 192) in the Data Appendix reveal that: 

• Relatively few of the fourth-grade public-school students (9 percent) did 
not read any books on their own outside of school in the month preceding 
the assessment; less than half (43 percent) read five or more books during 
the same period. 

• A smaller percentage of males than females read five or more books on 
their own outside of school during the month prior to the assessment; a 
greater percentage of males than females read no books. 

• Less than half of the White students (43 percent), about half of the Black 
students (46 percent), less than half of the Hispanic students (38 percent), 
and about half of the American Indian students (52 percent) read five or 
more books on their own outside of school. 

• Relatively few of the White students (8 percent), some of the Black 
students (11 percent), some of the Hispanic students (12 percent), and 
some of the American Indian students ( 1 6 percent) read no books on their 
own outside of school. 

• Average reading proficiency was lowest for students who read no books on 
their own outside of school during the month prior to the assessment. 
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TABLE 27 



Students’ Reports on the Number of 
Books Read Outside of School in the 
Past Month 



Arkansas 



Southeast 



Nation 
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The NAEP reading scale ranges from 0 to 500. The standard errors of the statistics appear in parentheses. It 
can be said with about 95 percent confidence that, for each population of interest, the value for the entire 
population is within ± 2 standard errors of the estimate for the sample. In comparing two estimates, one must 
use the standard error of the difference (see Appendix A for details). 
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Finally, regarding the frequency of taking books out of the school library or public library 
for their own enjoyment, from Table 28 and Table A28 (Page 194) in the Data Appendix: 

* In Arkansas, 17 percent of the fourth-grade public-school students took 
books out of the library for their own enjoyment almost every day; 

19 percent never or hardly ever did so. Across the nation, 15 percent took 
books out of the library for their own enjoyment almost every day and 
15 percent never or hardly ever did so. 

* About the same percentage of fourth-grade females (18 percent) as males 
(15 percent) in Arkansas took books out of the library for their own 
enjoyment almost every day. 

* A smaller percentage of fourth-grade females (14 percent) than males 
(24 percent) in Arkansas never or hardly ever took books out of the library 
for their own enjoyment. 

* Some of the White students (15 percent), about one quarter of the Black 
students (21 percent), some of the Hispanic students (18 percent), and 
some of the American Indian students (19 percent) in Arkansas took 
books out of the library for their own enjoyment almost every day. 

* Some of the White students (18 percent), about one quarter of the Black 
students (21 percent), about one quarter of the Hispanic students 
(22 percent), and some of the American Indian students (20 percent) in 
Arkansas never or hardly ever took books out of the library for their own 
enjoyment. 

* Students in Arkansas who took books out of the library almost every day 
had a higher average reading proficiency than students who never or hardly 
ever took books out of the library for their own enjoyment. 
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TABLE 28 



Students’ Reports on Taking Books 
Out of the Library 



Arkansas 



Southeast 



Nation 



How often do you take books out of the 
school library or public library for your 
own enjoyment? 

Almost every day 

Once or twice a week 

Once or twice a month 

Never or hardly ever 
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The NAEP reading scale ranges from 0 to 500. The standard errors of the statistics appear in parentheses. It 
can be said with about 95 percent confidence that, for each population of interest, the value for the entire 
population is within ± 2 standard errors of the estimate for the sample. In comparing two estimates, one must 
use the standard error of the difference (see Appendix A for details). 



READING IN THE HOME 

The presence of parents or siblings who model and share reading, and the availability of 
reading materials in the home are critical factors in the development of students’ 
appreciation of reading and, ultimately, their comprehension and fluency. 45 Children’s 
reading materials tend to consist of what is readily available to them. 46 Students 
participating in the Trial State Assessment were asked about the availability of newspapers, 
magazines, books, and an encyclopedia at home. They were also asked about the 
frequency with which they discussed things they read with friends and family. 



45 D. Taylor. Family Literacy : Young Children Learning to Read and Write. (Exeier, NH: Heinemann 
Educational Books, 1983). 

46 J. Ingham. Books and Reading Development: The Bradford Book Flood Experiment. (London: Heinemann 
Educational Books, 1981). 



THE 1992 NAEP TRIAL STATE ASSESSMENT 



103 



99 



Arkansas 



Average reading proficiency associated with having zero to two, three, or four of these types 
of materials in the home is shown in Table 29 and Table A29 (Page 196) in the Data 
Appendix. The data for Arkansas reveal that: 



• Students who had all four of these types of materials in the home showed 
a higher reading proficiency than did students with zero to two types of 
materials. Across the nation, students who had all four types of materials 
showed a higher reading proficiency than did students who had zero to two 
types. 

• Less than half of the White students (33 percent), about one quarter of the 
Black students (25 percent), about one quarter of the Hispanic students 
(21 percent), and about one quarter of the American Indian students 
(23 percent) had all four types of these reading materials in their homes. 

• Less than half of the White students (34 percent), less than half of the 
Black students (44 percent), less than half of the Hispanic students 
(41 percent), and about half of the American Indian students (47 percent) 
had zero to two types of these reading materials in their homes. 
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TABLE 29 



Students’ Reports on Types of 
Reading Materials in the Home 



Arkansas 



Southeast 



Nation 



Does your family have, or receive on a 
regular basis, any of the foiiowing items: 
more than 25 books, an encyclopedia, 
new papers, magazines? 



Zero to two types 
Three types 
Four types 
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The NAEP reading scale ranges from 0 to 500. The standard errors of the statistics appear in parentheses. It 
can be said with about 95 percent confidence that, for each population of interest, the value for the entire 
population is within ± 2 standard errors of the estimate for the sample. In comparing two estimates, one must 
use the standard error of the difference (see Appendix A for details). 
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Regarding the frequency of discussing with friends and family what the fourth-grade 
students read, Table 30 and Table A30 (Page 198) in the Data Appendix show 
that: 

• In Arkansas, 31 percent of the fourth-grade public-school students 
discussed with friends or family what they read almost every day; 
24 percent never or hardly ever discussed what they read. Across the 
nation, 27 percent discussed with friends or family what they read almost 
every day and 24 percent never or hardly ever discussed what they read. 

• About one quarter of the White students (28 percent), less than half of the 
Black students (39 percent), less than half of the Hispanic students 
(37 percent), and about one quarter of the American Indian students 
(29 percent) in Arkansas discussed with friends or family what they read 
almost every day. 

• About one quarter of the White students (24 percent), about one quarter 
of the Black students (24 percent), about one quarter of the Hispanic 
students (25 percent), and about one quarter of the American Indian 
students (28 percent) in Arkansas never or hardly ever discussed with 
friends or family what they read. 

• Students in Arkansas who discussed what they read with friends or family 
almost every day had about the same reading proficiency as students who 
never or hardly ever discussed with friends or family what they read. 
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table 30 | Students’ Reports on Talking With 

I Friends and Family About Reading 
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Nation 



How often do you talk with your 
friends or family about 
something you have read? 
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Never or hardly ever 



and 

Proficiency 

'' 3lTl.11 

: > 31 ( 0 . 9 ) 
216 (1.6) ", 

24 ( 1 . 0 ) / 

208 :{T •?)"■:■ 






< Percentage - 
Proficiency 

30(1.2) 

| 207 ( 2.6) 

33(2:2) ' 

218(5.0) , 



Percentage 

and 

Proficiencysi 

27(’:0.7);:::, : .v:: 




224 { 1.2) . 

\- v , ' ^ f ' ' 

■l 15(0.7),,, ££* 

;S : 

§.208 (15) t 



The NAEP reading scale ranges from 0 to 500. The standard errors of the statistics appear in parentheses. It 
can be said with about 95 percent confidence that, for each population of interest, the value for the entire 
population is within ± 2 standard errors of the estimate for the sample. In comparing two estimates, one must 
use the standard error of the difference (see Appendix A for details). 
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HOURS OF TELEVISION WATCHED PER DAY 



Many avid student readers watch a lot of television, while other children neither watch 
much television nor read. 47 However, despite these findings, television viewing has an 
effect on time given to reading — frequent television viewing limits the amount of time 
available for other activities such as reading 48 Students participating in the Trial State 
Assessment were asked to report on the amount of television they watched each day. 
Table 31 and Table A31 (Page 200) in the Data Appendix show that, in Arkansas: 

• Some of the fourth-grade public-school students (16 percent) watched one 
hour or less of television each day; about one quarter (28 percent) watched 
six hours or more. 

• A greater percentage of males than females tended to watch six or more 
hours of television daily. However, about the same percentage of males 
as females watched one hour or less per day. 

• About one quarter of the White students (21 percent), about half of the 
Black students (47 percent), about one quarter of the Hispanic students 
(30 percent), and less than half of the American Indian students 
(31 percent) watched six or more hours of television each day. 

• Some of the White students (17 percent), some of the Black students 
(11 percent), some of the Hispanic students (20 percent), and some of the 
American Indian students (14 percent) watched one hour or less per day. 

• Average reading proficiency was lowest for students who spent six hours 
or more watching television each day. 



47 S. Neuman. “The Home Environment and Fifth-grade Students’ Leisure Reading,” Elementary School 
Journal, 83. (1986). pp. 333-343. 

48 P. Heather. Young People's Reading: A Study of the Leisure Reading of 13-15 Year Olds. (Sheffield, 
England: University of Sheffield, Center for Research on User Studies, 1981). 
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TABLE 31 



Students’ Reports on the Amount of 
Time Spent Watching Television 
Each Day 



Arkansas 



Southeast 



Nation 




The NAEP reading scale ranges from 0 to 500. The standard errors of the statistics appear in parentheses. It 
can be said with about 95 percent confidence that, for each population of interest, the value for the entire 
population is within ± 2 standard errors of the estimate for the sample. In comparing two estimates, one must 
use the standard error of the difference (see Appendix A for details). 



SUMMARY 



Children who grow up in environments that support reading activities develop better 
reading skills. 49 Some out-of-school factors cannot be changed, but others can be altered 
in a positive way to influence a student’s learning and motivation. Partnerships among 
students, parents, teachers, and the larger community can affect the educational 
environment in the home, resulting in more out-of-school reading and an increased value 
placed on educational achievement, among other desirable outcomes. 



The data related to out-of-school factors show that: 

• In Arkansas, 39 percent of the fourth-grade public- school students 
reported that they read for fun almost every day while 16 percent never or 
hardly ever did so. 

• Relatively few of the fourth-grade public-school students in Arkansas 
(9 percent) did not read any books on their own outside of school during 
the month preceding the assessment; less than half (43 percent) read five 
or more books during the same period. 



49 Dolores Durkin. Children who Read Early . (New York: Teachers College Press, Columbia University, 
1966). 
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• In Arkansas, 17 percent of the fourth-grade public-school students took 
books out of the library for their own enjoyment almost every day; 
19 percent never or hardly ever did so. Across the nation, 15 percent took 
books out of the library for their own enjoyment almost every day and 
15 percent never or hardly ever did so. 

• Students in Arkansas who had four types of reading materials in the home 
(newspapers, magazines, more than 25 books, and an encyclopedia) 
showed a higher reading proficiency than did students with zero to two 
types of materials. Across the nation, students who had all four types of 
materials showed a higher reading proficiency than did students who had 
zero to two types. 

• In Arkansas, 31 percent of the fourth-grade public-school students 
discussed with friends or family what they read almost every day; 
24 percent never or hardly ever discussed what they read. Across the 
nation, 27 percent discussed with friends or family what they read almost 
every day and 24 percent never or hardly ever discussed what they read. 

• Some of the fourth-grade public-school students in Arkansas (16 percent) 
watched one hour or less of television each day; about one quarter 
(28 percent) watched six hours or more. Average reading proficiency in 
Arkansas was lowest for students who spent six hours or more watching 
television each day. 
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APPENDIX A 
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Procedural Appendix 

This appendix provides an overview of the technical details of the 1992 Trial State 
Assessment Program in reading. It includes a discussion of the history of NAEP, the 
assessment design, the reading framework and objectives upon which the assessment was 
based, and the procedures used to analyze the results. 



A Recent History of NAEP 

The National Assessment of Educational Progress (NAEP) is a Congressionally mandated 
project of the National Center for Education Statistics (NCES) that has collected and 
reported information for nearly 25 years on what American students know and what they 
can do. It is the nation’s only ongoing, comparable, and representative assessment of 
student achievement. Its assessments are given to scientific samples of youths attending 
both public and private schools and enrolled in grades four, eight, or twelve. 

In 1988, Congress authorized a new aspect of NAEP that allowed states and territories to 
participate voluntarily in a trial state assessment, using samples representative of their own 
students, to provide state-level data comparable to the nation and each of the other 
participating jurisdictions. Pursuant to that law, in 1990, the mathematics achievement of 
eighth graders was assessed in 40 jurisdictions (states, territories, and the District of 
Columbia). The results were reported in The State of Mathematics Achievement: NAEP’s 
1990 Assessment of the Nation and the Trial Assessment of the States. (Washington, DC: 
National Center for Education Statistics, 1991). 
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Over time there have been many changes in emphasis of NAEP assessment and reporting, 
both to take advantage of new technologies and to reflect changing trends in education. 

In 1984, a new technology called Item Response Theory (1RT) made it possible to create 
“scale scores” for NAEP similar to those the public was accustomed to seeing for the 
annual Scholastic Aptitude Test (SAT). Educational Testing Service, in its role as 
Government grantee carrying out NAEP operations, devised a new way to describe 
performance against this scale, called “anchor levels.” Starting in 1984, NAEP results were 
reported by “anchor levels.” Anchor levels describe distributions of performance at selected 
points along the NAEP scale (i.e., standard deviation units). Anchor levels show how 
groups of students perform relative to each other, but not whether this performance is 
adequate. 

This 1992 reading report marks NCES’s continued attempt to shift to standards-based 
reporting of National Assessment statistics. The transition is being made now to report 
NAEP results by “achievement levels.” Achievement levels describe how students should 
perform relative to a body of content reflected in the NAEP frameworks (i.e., how much 
students should know). The impetus for this shift lies in the belief that NAEP data will 
take on more meaning for the public if they show what proportion of our youth are able 
to meet standards of performance necessary for a changing world. For the 1992 reading 
assessment, an anchoring process was applied to these achievement levels in order to 
describe what students are able to do at each of these achievement levels. Chapter 1 of this 
report describes how the 1992 standards were prepared, provides examples of assessment 
questions that illustrate the reading content reflected in the descriptions of the NAEP 
achievement levels, and presents the results of student performance. 



Assessment Content 

The objectives for the assessment were developed through a consensus process managed 
by the Council of Chief State School Officers, and the items were developed through a 
similar process managed by Educational Testing Service. The development of the Trial 
State Assessment Program benefitted from the involvement of hundreds of representatives 
from State Education Agencies who attended numerous NETWORK meetings; served on 
committees; reviewed the framework, objectives, and questions; and, in general, provided 
important suggestions on all aspects of the program. 

The framework for the Trial State Assessment Program was developed using a broad-based 
consensus process, as described in the Overview to this report. 1 The reading assessment 
framework was a four-by-three matrix specifying three reading purposes: reading for 
literary experience, reading to be informed, and reading to perform a task. The reading to 
perform a task category was not evaluated and reported for grade 4. The four descriptions 
of reading stances include: Initial Understanding; Developing an Interpretation; Personal 
Reflection and Response; and Demonstrating a Critical Stance (see Figures A1 and A2). 



See National Assessment Governing Board Reading Framework for the 1992 National Assessment of 
Educational Progress. (Washington, DC: United States Department of Education, 1992) for a description of 
the frameworks and objectives. 
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figure Ai | Description of Reading Purposes 



Reading involves an interaction between a specific type of text or written material and a 
reader who typically has a purpose for reading that is related to the type of text and the 
context of the reading situation. The 1992 NAEP reading assessment presented three types 
of text to students representing each of three reading purposes: literary text for literary 
experience, informational text to gain information, and documents to perform a task. At 
grade 4, only the first two reading purposes were assessed. Students’ reading abilities were 
evaluated in terms of a single purpose for each type of text. 
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Reading for Literary Experience 



Reading for literary experience involves reading literary text to explore the human condition, to relate 
narrative events with personal experience, and to consider the interplay in the selection among emotions, 
events, and possibilities. Students in the NAEP reading assessment were provided with a wide variety of 
literary text such as short stories, poems, fables, historical fiction, science fiction, and mysteries. 



Reading to Gain Information 



Reading to gain information involves reading informative passages in- order to obtain some general or 
specific information. This often requires a more utilitarian approach to reading that requires the use of 
certain reading/thinking strategies different from those used for other purposes. In addition, reading to 
gain information often involves reading and interpreting adjunct aids such as charts, graphs, maps, and 
tables that provide supplemental or tangential data. Informational passages in the NAEP reading 
assessment included biographies, science articles, encyclopedia entries, primary and secondary historical 
accounts, and newspaper editorials. 



Reading to Perform a Task 



Reading to perform a task involves reading various types of materials for the purpose of applying the 
information or directions in completing a specific task. The reader’s purpose for gaining meaning extends 
beyond understanding the text to include the accomplishment of a certain activity. Documents requiring 
students in the NAEP reading assessment to perform a task included directions for creating a time capsule, 
instructions on how to write a letter to your Senator, a bus schedule, and a tax form. In 1992, reading to 
perform a task was assessed only at grades 8 and 12. 
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Trial State Assessment 



Readers interact with text in various ways as they use background knowledge and 
understanding of text to construct, extend, and examine meaning. The NAEP reading 
assessment framework specified four reading stances to be assessed that represent various 
interactions between readers and texts. These stances are not meant to describe a hierarchy 
of skills or abilities. Rather, they are intended to describe behaviors that readers at all 
developmental levels should exhibit. 



Initial Understanding 



Initial understanding requires a broad, preliminary construction of an understanding of the text. Questions 
testing this aspect ask the reader to provide an initial impression or unreflected understanding of what was 
read. In the 1992 NAEP reading assessment, the first question following a passage was usually one testing 
initial understanding. 



Developing an Interpretation 



Developing an interpretation requires the reader to go beyond the initial impression to develop a more 
complete understanding of what, was read. Questions testing this aspect require a more specific 
understanding of the text and involve linking information across parts of the text as well as focusing on 
specific information. 



Personal Reflection and Response 



Personal response requires the reader to connect knowledge from the text more extensively with his or her 
own personal background knowledge and experience. The focus is on how the text relates to personal 
experience, and questions on this aspect ask the readers to reflect and respond from a personal 
perspective. For the 1992 NAEP reading assessment, personal response questions were typically 
formatted as constructed-response items to allow for individual possibilities and varied responses. 



Demonstrating a Critical Stance 



Demonstrating a critical stance requires the reader to stand apart from the text, consider it, and judge it 
objectively. Questions on this aspect require the reader to perform a variety of tasks such as critical 
evaluation, comparing and contrasting, applications to practical tasks, and understanding the impact of such 
text features as irony, humor, and organization. These questions focus on the reader as interpreter/critic 
and require reflection and judgments to be made by the reader. 
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Assessment Design 

The 1992 reading assessment was based on a focused balanced incomplete block (BIB) 
spiral matrix design — a design that enables broad coverage of reading content while 
minimizing the burden for any one student. The 1992 NAEP reading assessment for 
grades 4, 8, and 12 contained a total of 170 discrete constructed-response items, 135 of 
which were regular constructed-response items and 35 of which were extended 
constructed-response items. 

In the Trial State reading assessment at grade 4, 85 reading items were developed for the 
assessment, including 35 regular constructed-response items, eight extended 
constructed-response items, and 42 multiple-choice items. 

The first step in implementing the BIB design required selecting grade-appropriate passages 
and developing questions to assess the four reading stances specified in the framework. 

The questions were assembled into units called blocks , with each block designed to be 
completed in 25 or 50 minutes. At grade 4, eight blocks were designed; they required 
25 minutes of student time for completion. The blocks were assembled into assessment 
booklets so that each booklet contained three background questionnaires — the first 
consisting of general background questions, the second comprising reading background 
questions, and the third containing questions about the students’ motivation to do well in 
the assessment — and two blocks of cognitive reading items. The questions in the first 
section were read aloud to the students, usually taking about 10 minutes to complete. 
Students were then given 50 minutes to complete two 25-minute blocks of reading items, 
five minutes to complete the second background questionnaire, and three minutes to 
complete the third background questionnaire. Thus, the assessment required slightly over 
one hour of student time. 

In accordance with the BIB design, the blocks were assigned to the assessment booklets so 
that there were a total of 16 booklets at grade 4. Blocks appeared in both positions within 
a booklet and were paired with blocks assessing the same purpose for reading as well as 
blocks assessing other purposes. The booklets were spiraled or interleaved in a systematic 
sequence so that each booklet appeared an appropriate number of times in the sample. 
The students within an assessment session were assigned booklets in the order in which the 
booklets were spiraled. Thus, students in any given session received a variety of different 
booklets and only a small number of students in the session received the same booklet. 
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Data Analysis and Scales 

Once the assessments were conducted and information from the assessment booklets had 
been compiled in a database, the assessment data were weighted to match known 
population proportions and adjusted for nonresponse. Analyses were then conducted to 
determine the percentages of students who gave various responses to each cognitive and 
background question. 

Item response theory (IRT) was used to estimate average reading proficiency for each 
jurisdiction and for various subpopulations, based on students’ performance on the set of 
reading items they received. IRT provides a common scale on which performance can be 
reported for the nation, each jurisdiction, and subpopulations, even when all students do 
not answer the same set of questions. This common scale makes it possible to report on 
relationships between students’ characteristics (based on their responses to the background 
questions) and their overall performance on the assessment. 

A scale ranging from 0 to 500 was created to report performance for each of the two reading 
purposes at grade 4 (reading for literary experience and reading to gain information). The 
scales summarize examinee performance across all three item types used in the assessment 
(multiple-choice, regular constructed-response, and extended constructed-response). In 
producing the scales, three distinct IRT models were used. Multiple-choice items were 
scaled using the three-parameter logistic model; regular constructed-response items were 
scaled using the two-parameter logistic model; and the extended constructed-response items 
were scaled using a generalized partial-credit model. Each reading purpose scale was based 
on the distribution of student performance across the grades assessed in the 1992 national 
assessment (grades 4, 8, and 12) and had a mean of 250 and a standard deviation of 50. 

A composite scale was created as an overall measure of students’ reading proficiency. At 
grade 4, the composite scale was a weighted average of the two reading purpose scales, 
where the weight for each reading purpose was proportional to the relative importance 
assigned to that purpose in the specifications developed by the Reading Objectives Panel 
(55 percent literary experience and 45 percent gain information). 



Questionnaires for Teachers and Schools 

As part of the Trial State Assessment, questionnaires were given to the reading teachers of 
assessed students and to the principal or other administrator in each participating school. 

A Background Panel drafted a set of issues and guidelines and made recommendations 
concerning the design of these questionnaires. For the 1992 assessment, the teacher and 
school questionnaires focused on five educational areas: instructional content, instructional 
practices and experiences, teacher characteristics, school conditions and context, and 
conditions beyond school (i.e., home support, out-of-school activities, and attitudes). 
Similar to the development of the materials given to students, the guidelines and the teacher 
and school questionnaires were prepared through an iterative process that involved 
extensive development, field testing, and review by external advisory groups. 
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It is important to note that in this report, as in all NAEP reports, the student is always the 
unit of analysis, even when information from the teacher or school questionnaire is being 
reported. Having the student as the unit of analysis makes it possible to describe the 
instruction received by representative samples of fourth-grade students in public schools. 
Although this approach may provide a different perspective from that which would be 
obtained by simply collecting information from a sample of fourth- grade reading teachers 
or from a sample of schools, it is consistent with NAEP’s goal of providing information 
about the educational context and performance of students. 



READING TEACHER QUESTIONNAIRE 

The questionnaires for fourth-grade teachers consisted of two parts. The first requested 
information about the teacher, such as race/ethnicity and gender, as well as academic 
degrees held, teaching certification, training in reading, and ability to get instructional 
resources. In the second part, teachers were asked to provide information on each class 
they taught that included one or more students who participated in the Trial State 
Assessment Program. The information included, among other things, the extent to which 
worksheets or workbooks were used, the emphasis placed on different reading skills, and 
various instructional approaches. Because of the nature of the sampling for the Trial State 
Assessment, the responses to the reading teacher questionnaire do not necessarily represent 
all fourth-grade reading teachers in a state or territory. Rather, they represent the teachers 
of the particular students being assessed. 



SCHOOL CHARACTERISTICS AND POLICIES QUESTIONNAIRE 

An extensive school questionnaire was completed by principals or other administrators in 
the schools participating in the Trial State Assessment. In addition to questions about the 
individuals who completed the questionnaires, there were questions about school policies, 
course offerings, and special priority areas, among other topics. 



Estimating Variability 

The statistics reported by NAEP (average proficiencies, percentages of students at or above 
particular achievement levels, and percentages of students responding in certain ways to 
background questions) are estimates of the corresponding information for the population 
of fourth-grade students in public schools in a state. These estimates are based on the 
performance of carefully selected, representative samples of fourth-grade public-school 
students from the state or territory. 

If a different representative sample of students were selected and the assessment repeated, 
it is likely that the estimates might vary somewhat, and both of these sample estimates 
might differ somewhat from the value of the mean or percentage that would be obtained 
if every fourth-grade public-school student in the state or territory were assessed. Virtually 
all statistics that are based on samples (including those in NAEP) are subject to a certain 
degree of uncertainty. The uncertainty attributable to using samples of students is referred 
to as sampling error . 
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Like almost all estimates based on assessment measures, NAEP’s total group and subgroup 
proficiency estimates are subject to a second source of uncertainty, in addition to sampling 
error. As previously noted, each student who participated in the Trial State Assessment 
was administered a subset of questions from the total set of questions. If each student had 
been administered a different, but equally appropriate, set of the assessment questions - 
or the entire set of questions — somewhat different estimates of total group and subgroup 
proficiency might have been obtained. Thus, a second source of uncertainty arises because 
each student was administered a subset of the total pool of questions. 

In addition to reporting estimates of average proficiencies, proportions of students at or 
above particular achievement levels, and proportions of students giving various responses 
to background questions, this report also provides estimates of the magnitude of the 
uncertainty associated with these statistics. These measures of the uncertainty are called 
standard errors and are given in parentheses in each of the tables in the report. The 
standard errors of the estimates of reading proficiency statistics reflect both sources of 
uncertainty discussed above. The standard errors of the other statistics (such as the 
proportion of students answering a background question in a certain way or the proportion 
of students in certain racial/ethnic groups) reflect only sampling error. NAEP uses a 
methodology called the jackknife procedure to estimate these standard errors. 

The reader is reminded that, as in all surveys, NAEP results are also subject to other kinds 
of errors including the effects of necessarily imperfect adjustment for student and school 
nonresponse and other largely unknowable effects associated with the particular 
instrumentation and data collection methods used. Nonsampling errors can be attributed 
to a number of sources: inability to obtain complete information about all selected 
students in all selected schools in the sample (some students or schools refused to 
participate, or students participated but answered only certain items); ambiguous 
definitions; differences in interpreting questions; inability or unwillingness to give correct 
information; mistakes in recording, coding, or scoring data; and other errors of collecting, 
processing, sampling, and estimating missing data. The extent of nonsampling errors is 
difficult to estimate. By their nature, the impact of such errors cannot be reflected in the 
data-based estimates of uncertainty provided in NAEP reports. 



Drawing Inferences from the Results 

One of the goals of the Trial State Assessment Program is to make inferences about the 
overall population of fourth-grade students in public schools in each participating state and 
territory based on the particular sample of students assessed. The results from the sample 
-- taking into account the uncertainty associated with all samples — are used to make 
inferences about the population. 

The use of confidence intervals , based on the standard errors, provides a way to make 
inferences about the population means and proportions in a manner that reflects the 
uncertainty associated with the sample estimates. An estimated sample mean proficiency 
± 2 standard errors approximates a 95 percent confidence interval for the corresponding 
population quantity. This means that with approximately 95 percent confidence, the 
average performance of the entire population of interest (e.g., all fourth-grade students in 
public schools in a state or territory) is within ± 2 standard errors of the sample mean. 
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As an example, suppose that the average reading proficiency of the students in a particular 
state’s fourth-grade sample were 256 with a standard error of 1.2. A 95 percent confidence 
interval for the population quantity would be as follows: 

Mean ± 2 standard errors = 256 ± 2 * (1.2) = 256 ± 2.4 = 

256 - 2.4 and 256 + 2.4 = (253.6, 258.4) 

Thus, one can conclude with 95 percent confidence that the average proficiency for the 
entire population of fourth-grade students in public schools in that state is between 253.6 
and 258.4. 

Similar confidence intervals can be constructed for percentages, provided that the 
percentages are not extremely large (greater than 90 percent) or extremely small (less than 
10 percent). For extreme percentages, confidence intervals constructed in the above 
manner may not be appropriate and procedures for obtaining accurate confidence intervals 
are quite complicated. 



Analyzing Subgroup Differences in Proficiencies and Proportions 

In addition to the overall results, this report presents outcomes separately for a variety of 
important subgroups. Many of these subgroups are defined by shared characteristics of 
students, such as their gender, race/ethnicity, and the type of community in which their 
school is located. Other subgroups are defined by students’ responses to background 
questions. Still other subgroups are defined by the responses of the assessed students’ 
reading teachers to questions in the reading teacher questionnaire. 

In Chapter 1 of this report, differences between the state and nation were tested for overall 
reading proficiency, percent of students at or above the Proficient level, and for each of the 
purposes for reading. In Chapter 2, significance tests were conducted for the overall 
proficiency for each of the subpopulations, as well as percent of students at or above the 
Proficient level for males and females. In Chapters 3-6, comparisons were made across 
subgroups for responses made to various background questions. 

As an example, consider the question: Do students who reported reading three or four books 
outside of school each month exhibit higher average reading proficiency than students who 
reported reading no books outside of school? 

To answer the question posed above, begin by comparing the average reading proficiency 
for the two groups being analyzed. If the mean for the group that reported reading three 
or four books outside of school is higher, it may be tempting to conclude that that group 
does have higher achievement than the group that reported reading no books outside of 
school. However, even though the means differ, there may be no real difference in 
performance between the two groups in the population because of the uncertainty 
associated with the estimated average proficiency of the groups in the sample. Remember 
that the intent is to make a statement about the entire population, not about the particular 
sample that was assessed. The data from the sample are used to make inferences about the 
population as a whole. 



117 



THE 1992 NAEP TRIAL STATE ASSESSMENT 



113 



Arkansas 



As discussed in the previous section, each estimated sample mean proficiency (or 
proportion) has a degree of uncertainty associated with it. It is therefore possible that if 
all students in the population had been assessed, rather than a sample of students, or if the 
assessment had been repeated with a different sample of students or a different, but 
equivalent, set of questions, the performances of various groups would have been different. 
Thus, to determine whether there is a real difference between the mean proficiency (or 
proportion of a certain attribute) for two groups in the population, an estimate of the 
degree of uncertainty associated with the difference between the proficiency means or 
proportions of those groups must be obtained for the sample. This estimate of the degree 
of uncertainty — called the standard error of the difference between the groups — is obtained 
by taking the square of each group’s standard error, summing these squared standard errors, 
and then taking the square root of this sum. 

Similar to the manner in which the standard error for an individual group mean or 
proportion is used, the standard error of the difference can be used to help determine 
whether differences between groups in the population are real. The difference between the 
mean proficiency or proportion of the two groups ± 2 standard errors of the difference 
represents an approximate 95 percent confidence interval. If the resulting interval includes 
zero, there is insufficient evidence to claim a real difference between groups in the 
population. If the interval does not contain zero, the difference between groups is 
statistically significant (different) at the .05 level. 

As an example, to determine whether the average reading proficiency of fourth-grade 
females is higher than that of fourth-grade males in a particular state’s public schools, 
suppose that the sample estimates of the mean proficiencies and standard errors for females 
and males were as follows: 



Group 


Average 


Standard 


Proficiency 


Error 


Female 


259 


2.0 


Male 


255 


2.1 



The difference between the estimates of the mean proficiencies of females and males is four 
points (259 - 255). The standard error of this difference is 

V 2.0 2 + 2.1 2 = 2.9 

Thus, an approximate 95 percent confidence interval for this difference is 

Mean difference ± 2 standard errors of the difference = 

4 ± 2 * (2.9) = 4 ± 5.8 = 4 - 5.8 and 4 + 5.8 = (-1.8, 9.8) 

The value zero is within this confidence interval, which extends from -1.8 to 9.8 (i.e., zero 
is between -1.8 and 9.8). Thus, there is insufficient evidence to claim a difference in average 
reading proficiency between the populations of fourth-grade females and males in public 
schools in the state. 2 



2 The procedure described above (especially the estimation of the standard error of the difference) is, in a strict 
sense, only appropriate when the statistics being compared come from independent samples. For certain 
comparisons in the report, the groups were not independent. In those cases, a different (and more 
appropriate) estimate of the standard error of 'the difference was used. 
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Throughout this report, when the mean proficiencies or proportions for two groups were 
compared, procedures like the one described above were used to draw the conclusions that 
are presented. If a statement appears in the report indicating that a particular group had 
higher ( or lower) average proficiency than a second group, the 95 percent confidence 
interval for the difference between groups did not contain zero. When a statement indicates 
that the average proficiency or proportion of some attribute was about the same for two 
groups, the confidence interval included zero, and thus no difference could be assumed 
between the groups. The reader is cautioned to avoid drawing conclusions solely on the 
basis of the magnitude of the differences. A difference between two groups in the sample 
that appears to be slight may represent a statistically significant difference in the population 
because of the magnitude of the standard errors. Conversely, a difference that appears to 
be large may not be statistically significant. 

The procedures described in this section, and the certainty ascribed to intervals (e.g., a 
95 percent confidence interval), are based on statistical theory that assumes that only one 
confidence interval or test of statistical significance is being performed. However, in each 
chapter of this report, many different groups are being compared (i.e., multiple sets of 
confidence intervals are being analyzed). In sets of confidence intervals, statistical theory 
indicates that the certainty associated with the entire set of intervals is less than that 
attributable to each individual comparison from the set. To hold the certainty level for the 
set of comparisons at a particular level (e.g., .95), adjustments (called multiple comparison 
procedures) must be made to the methods described in the previous section. One such 
procedure — the Bonferroni method — was used in the analyses described in this report to 
form confidence intervals for the differences between groups whenever sets of comparisons 
were considered. Thus, the confidence intervals in the text that are based on sets of 
comparisons are more conservative than those described on the previous pages. A more 
detailed description of the use of the Bonferroni procedure appears in the Trial State 
Assessment technical report. 



Statistics with Poorly Determined Standard Errors 

The standard errors for means and proportions reported by NAEP are statistics and 
therefore are subject to a certain degree of uncertainty. In certain cases, typically when the 
standard error is based on a small number of students, or when the group of students is 
enrolled in a number of schools, the amount of uncertainty associated with the standard 
errors may be quite large. Throughout this report, estimates of standard errors subject to 
a large degree of uncertainty are followed by the symbol In such cases, the standard 
errors — and any confidence intervals or significance tests involving these standard errors 
— should be interpreted cautiously. Further details concerning procedures for identifying 
such standard errors are discussed in the Trial State Assessment technical report. 
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Minimum Subgroup Sample Sizes 

Results for reading proficiency and background variables were tabulated and reported for 
groups defined by race/ethnicity, type of community, gender, and parents’ education level. 
NAEP collects data for five racial/ethnic subgroups (White, Black, Hispanic, Asian/Pacific 
Islander, and American Indian/Alaskan Native), four types of communities (Advantaged 
Urban, Disadvantaged Urban, Extreme Rural, and Other Communities), and five levels of 
parents’ education (Graduated College, Some Education After High School, Graduated 
High School, Did Not Finish High School, and I Don’t Know). However, in many states 
or territories, and for some regions of the country, the number of students in some of these 
groups was not sufficiently high to permit accurate estimation of proficiency and/or 
background variable results. As a result, data are not provided for the subgroups with very 
small sample sizes. For results to be reported for any subgroup, a minimum sample of 
62 students was required. For statistical tests pertaining to subgroups, the sample size for 
both groups had to be at least 62. This number was determined by computing the sample 
size required to detect an effect size of .2 total-group standard deviation units with a 
probability of .8 or greater. 

The effect size of .2 pertains to the true difference between the average proficiency of the 
subgroup in question and the average proficiency for the total fourth-grade public-school 
population in the state or territory, divided by the standard deviation of the proficiency in 
the total population. If the true difference between subgroup and total group mean is 
.2 total-group standard deviation units, then a sample size of at least 62 is required to detect 
such a difference with a probability of .8. Further details about the procedure for 
determining minimum sample size appear in the Trial State Assessment technical report. 



Describing the Size of Percentages 

Some of the percentages reported in the text of the report are given quantitative 
descriptions. For example, the number of students being taught by teachers with master’s 
degrees in education might be described as “relatively few” or “almost all,” depending on 
the size of the percentage in question. Any convention for choosing descriptive terms for 
the magnitude of percentages is to some degree arbitrary. The descriptive phrases used in 
the report and the rules used to select them are shown below. 



Percentage 


Description of Text in Report 


p = 0 


None 


0 < p < 10 


Relatively few 


10 < p < 20 


Some 


20 < p < 30 


About one quarter 


30 < p < 44 


Less than half 


I 44 < p < 55 


About half 


55 < p < 69 


More than half 


69 < p < 79 


About three quarters 


79 < p < 89 


Many 


89 < p < 100 


Almost all 


o 

o 

II 

CL 


All 
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Reading Stimuli and Example Items 



This appendix contains replications of two of the eight reading passages used as the stimuli 
at grade 4. In addition, examples of two extended constructed-response items are 
presented, along with their scoring guides. The extended constructed-response questions, 
which required students to demonstrate more complex reading processes and understanding 
of the text by providing an extended response of a paragraph or more to the prompt, were 
scored using a four-point partial-credit model. For the extended constructed-response 
questions, students were given an entire blank (lined) page on which to provide their 
responses. Table A1 shows the percentages of students in Arkansas and the nation 
attaining each of the score levels for the two extended constructed-response example items. 
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TABLE A1 



Student Score-Level Percentages for 
Extended Constructed- Response 
Example Items at Grade 4 



Trial State Assessment 



SYBIL SOUNDS THE ALARM 
Example Item 1 

Arkansas 

Nation 

AMANDA CLEMENT 
Example Item 2 

Arkansas 

Nation 
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SYBIL SOUNDS 
THE ALARM 

by Drollene P. Brown 



A red sky at night does not usually 
cause wonder. But on the evening of 
April 26, 1777, the residents of 

Ludingtons’ Mills were concerned. The 
crimson glow was in the east, not from 
the west where the sun was setting. 

The Ludington family sat at supper, 
each one glancing now and again toward 
the eastern window. Sybil, at sixteen the 
oldest of eight children, could read the 
question in her mother’s worried eyes. 
Would Henry Ludington have to go 
away again? As commander of the only 
colonial army regiment between 
Danbury, Connecticut, and Peekskill, 
New York, Sybil’s father did not have 
much time to be with his family. 

Thudding hooves in the yard abruptly 
ended their meal. The colonel pushed 
back his chair and strode to the door. 
Although Sybil followed him with her 



eyes, she dutifully began to help her sister 
Rebecca clear the table. 

The girls were washing dishes when 
their father burst back into the room with 
a courier at his side. 

“Here, Seth,” said the colonel, “sit you 
down and have some supper. Rebecca, 
see to our weary friend.” 

Sybil, glancing over her shoulder, saw 
that the stranger was no older than she. 
A familiar flame of indignation burned 
her cheeks. Being a girl kept her from 
being a soldier! 

Across the room, her parents were 
talking together in low tones. Her 
father’s voice rose. 

“Sybil, leave the dishes and come 
here,” he said. 

Obeying quickly, she overheard her 
father as he again spoke to her mother. 
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“Abigail, she is a skilled rider. It is 
Sybil who has trained Star, and the horse 
will obey her like no other.” 

“That red glow in the sky,” Colonel 
Ludington said, turning now to his 
daughter, “is from Danbury. It’s been 
burned by British raiders. There are 
about two thousand Redcoats, and 
they’re heading for Ridgefield. Someone 
must tell our men that the lull in the 
fighting is over; they will have to leave 
their families and crops again.” 

“I’ll go! Star and I can do it!” Sybil 
exclaimed. She faced her mother. “Star 
is sure of foot, and will carry me safely.” 
“There are dangers other than slip- 



pery paths,” her mother said, softly. 
“Outlaws or deserters or Tories or even 
British soldiers may be met. You must 
be wary in a way that Star cannot.” 

A lump rose in Sybil’s throat. “I can 
do it,” she declared. 

Without another word, Abigail 
Ludington turned to fetch a woolen cape 
to protect her daughter from the wind 
and rain. One of the boys was sent to 
saddle Star, and Sybil was soon ready. 
When she had swung up on her sturdy 
horse, the colonel placed a stick in her 
hand. 

As though reciting an oath, she 
repeated her father’s directions: “Go 

south by the river, then along Horse 



O 
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Pond Road to Mohopac Pond. From 
there, turn right to Red Mills, then go 
north to Stormville.” The colonel stood 
back and saluted. She was off! 

At the first few isolated houses, 
windows or doors flew open as she 
approached. She shouted her message 
and rode on. By the time she reached the 
first hamlet, all was dark. There were 
many small houses there at the edge of 
Shaw’s Road, but everyone was in bed. 
Lights had not flared up at the sound of 
Star’s hoofbeats. Sybil had not 
anticipated this. Biting her lower lip, she 
pulled Star to a halt. After considering 
for a moment, she nudged the horse 
forward, and riding up to one cottage 
after another, beat on each door with her 
stick. 

“Look at the sky!” she shouted. 
“Danbury’s burning! All men muster at 
Ludingtons’!’’ 

At each village or cluster of houses, 
she repeated the cry. When lights began 
to shine and people were yelling and 
moving about, she would spur her horse 
onward. Before she and Star melted into 
the night, the village bells would be 
pealing out the alarm. 

Paths were slippery with mud and wet 
stones, and the terrain was often hilly and 
wooded. Sybil’s ears strained for sounds 
of other riders who might try to steal her 
horse or stop her mission. Twice she 
pulled Star off the path while unknown 



riders passed within a few feet. Both 
times, her fright dried her mouth and 
made her hands tremble. 

By the time they reached Stormville, 
Star had stumbled several times, and 
Sybil’s voice was almost gone. The 
town’s call to arms was sounding as they 
turned homeward. Covered with mud, 
tired beyond belief, Sybil could barely 
stay on Star’s back when they rode into 
their yard. She had ridden more than 
thirty miles that night. 

In a daze, she saw the red sky in the 
east. It was the dawn. Several hundred 
men were milling about. She had roused 
them in time, and Ludington’s regiment 
marched out to join the Connecticut 
militia in routing the British at 
Ridgefield, driving them back to their 
ships on Long Island Sound. 

Afterward, General George 
Washington made a personal visit to 
Ludingtons’ Mills to thank Sybil for her 
courageous deed. Statesman Alexander 
Hamilton wrote her a letter of praise. 

Two centuries later visitors to the area 
of Patterson, New York, can still follow 
Sybil’s route. A statue of Sybil on 
horseback stands at Lake Gleneida in 
Carmel, New York, and people in that 
area know well the heroism of Sybil 
Ludington. In 1978, a commemorative 
postage stamp was issued in her honor, 
bringing national attention to the heroic 
young girl who rode for independence. 



From Cobblestone’s September, 1983, issue: 
“Patriotic Tales of the American 
Revolution.” Copyright 1983, Cobblestone 
Publishing Inc.. Peterborough, NH 03548. 
Reprinted by permission of the publisher. 
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EXAMPLE ITEM l | Extended Constructed- Response Item 

I Sybil Sounds the Alarm 



f;i What are the imajor events in the story? Mi 
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General Scoring Rubric 




Demonstrates an understand! 


ng of an historical narrative by summarizing 


_ ; • . ( ; 1.1 * . 1 * ff 1 I : 1 • v : : 1 1 % . CJ J W 1 : Kjt W Vr J.H >3 * . . : : 


\!v^ / ‘ c * (t< 1 'i‘‘ " * « , ' (< ‘ 



Scoring Guide 



Unsatisfactory These , responses demonstrate little or no understanding of the 
SvbiPs ride hv brovidiinB bits of information from the storv. 

Partial These responses demonstrate some understanding of Sybil’s ride by 





Essential These responses demonstrate an understanding of at least two of the 
major events surrounding Sybil’s ride by providing a detailed account of these 



Extensive . These responses demonstrate an in-depth understanding of the 
: i rnajor , event s surroun ding Sybil's ride by providing a detailed: amount of major 
:;;: ||jle vents'; accompanied by an ^explain^ipp | significance Jg|The responses!;® 

. ../.d isplay, atJ^rojigh 'understanding of the story as a 
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Amanda 


••• 


Clement: 


IBB Hng^ r tjg 


The Umpire 


H WBk. ^ 

|£i&* * . ’• ; 


in a Skirt 


til - Mkobkosss^. 


Marilyn Kratz 









I T was A hot Sunday afternoon in Hawarden, a small town in western Iowa. 

Amanda Clement was sixteen years old. She sat quietly in the grandstand with her 
mother, but she imagined herself right out there on the baseball diamond with the players. 
Back home in Hudson, South Dakota, her brother Hank and his friends often asked her 
to umpire games. Sometimes she was even allowed to play first base. 

Today, Mandy, as she was called, could only sit and watch Hank pitch for Renville 
against Hawarden. The year was 1904, and girls were not supposed to participate in sports. 
But when the umpire for the preliminary game between two local teams didn’t arrive, Hank 
asked Mandy to make the calls. 
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Mrs. Clement didn’t want her daughter to umpire a public event, but at last Hank and 
Mandy persuaded her to give her consent. Mandy eagerly took her position behind the 
pitcher’s mound. Because only one umpire was used in those days, she had to call plays 
on four bases as well as strikes and balls. 

Mandy was five feet ten inches tall and looked very impressive as she accurately called 
the plays. She did so well that the players for the big game asked her to umpire for them 
-- with pay! 

Mrs. Clement was shocked at that idea. But Mandy finally persuaded her mother to 
allow her to do it. Amanda Clement became the first paid woman baseball umpire on 
record. 

Mandy’s fame spread quickly. Before long, she was umpiring games in North and South 
Dakota, Iowa, Minnesota, and Nebraska. Flyers, sent out to announce upcoming games, 
called Mandy the “World Champion Woman Umpire.” Her uniform was a long blue skirt, 
a black necktie, and a white blouse with UMPS stenciled across the front. Mandy kept 
her long dark hair tucked inside a peaked cap. She commanded respect and attention — 
players never said, “Kill the umpire!” They argued more politely, asking, “Beg your 
pardon, Miss Umpire, but wasn’t that a bit high?” 

Mandy is recognized in the Baseball Hall of Fame in Cooperstown, New York; the 
Women’s Sports Hall of Fame; and the Women’s Sports Foundation in San Francisco, 
California. In 1912, she held the world record for a woman throwing a baseball: 279 feet. 

Mandy’s earnings for her work as an umpire came in especially handy. She put herself 
through college and became a teacher and coach, organizing teams and encouraging athletes 
wherever she lived. Mandy died in 1971. People who knew her remember her for her work 
as an umpire, teacher, and coach, and because she loved helping people as much as she 
loved sports. 



“Amanda Clement: The Umpire in a Skirt” by Marilyn 
Kratz. Copyright 1987 by Marilyn Kratz. Copyright 1987 
by Carus Corporation. Reprinted by permission. 
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EXAMPLE ITEM 2 | Extended Constructed-Response Item 

I Amanda Clement: The Umpire in a Skirt 



; • Question , .."f f \ f,“‘ ,“ v v $ ( , (V 

If she were alive today, what question would you like to ask Mandy 
If:, about her career?; Explain why .the answer tqlyour question would?;-: || 
§gmbe importahtito 



i |i|| 

f ersolli Res pohse ! 



General Storing Rubric 

f Demonstrates an! understand i ng : of Aman da’s career as a baseball urrvpire' 
ahi|irpersonal;: i reartion,ito 



Scoring Guide 



Unsatisfactory . These responses demonstrate tittle or no understanding by 
|:)f^pirbyi t ding inappropriate (details or isolated bits of tnrorm^ioti, fmm fhe passage;^. 1 
or they pose a question that is unrelated to Mandy ’s career as a woman in a role 
traditionally dominated by males. Some responses may simply refer to 




Partial These responses demonstrate only partial understanding of Mandy -s-’. 
’ ‘7^^ree r t 'mid its nontraditional nature by posing one" question that is , either not ; 





| |C| question that irrelevant to the student 1 ?; own "understanding of what it is like 
to be an athlete who is highly successful or the first person to do something. 

I/” «li 8: «j= E^tjmsive ^li;i;-nr^^ :lf^t>otiises Contain one question that is explained mireL^ioif 
to a personal view of The passage and indicates the student has considered the 
more .abstract aspects ; ; qf the passage (e!gi;;; emotional considerations/personal 
fl challenges, societal reactions). These responses contain questions about issues 
!l!!|or reactions that; have " grown out of a careful consideration of the potential 
c c f - problems Mandy faced and the historical context in which she lived. : \ 
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APPENDIX C 
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Trial Stale Assessment 




Setting the Achievement Levels 



Setting achievement levels is a method for setting standards on the NAEP assessment that 
identifies what students should know and should be able to do at various points along the 
proficiency scale. The method depends on securing and summarizing a set of judgmental 
ratings of expectations for student educational performance on specific items. The NAEP 
proficiency scale is a numerical index of students’ performance in reading ranging from 
0 to 500 and has three achievement levels — Basic, Proficient, and Advanced — mapped 
onto it for each grade level assessed. 



In developing the threshold values for the levels, a broadly constituted panel of judges — 
including teachers (50 percent), non-teacher educators (20 percent), and non-educators 
(30 percent) — rated a grade-specific item pool using the Board’s policy definitions for 
Basic, Proficient, and Advanced. 1 The policy definitions were operationalized by the 
judges in terms of specific reading skills, knowledge, and behaviors that were judged to be 
appropriate expectations for students in each grade, and were in accordance with the 
current reading assessment framework. The policy definitions are as follows: 



BASIC 



PROFICIENT 



ADVANCED 



This level, below Proficient, denotes partial mastery of the knowledge and skills 
that are fundamental for proficient work at each grade — 4, 8, and 12. 

This central level represents solid academic performance for each grade tested — 
4, 8, and 12. Students reaching this level have demonstrated competency over 
challenging subject matter and are well prepared for the next level of schooling. 

This higher level signifies superior performance beyond proficient grade-level 
mastery at grades 4, 8, and 12. 



The judges’ operationalized definitions were incorporated into lists of descriptors that 
represented what borderline students should be able to do at each of the policy levels. The 
purpose of having panelists develop their own operational definitions of the achievement 
levels was to ensure that all panelists would have a common understanding of borderline 
performances and a common set of content-based referents to use during the item-rating 
process. 



1 Non-educators represented business, labor, governmeni service, parents, and the general public. 
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For the multiple-choice and short constructed-response items that were scored 
correct /incorrect, the judges (22 at grade 4) each rated half of the items in the NAEP pool 
in terms of the expected probability that a student at a borderline achievement level would 
answer the item correctly, based on the judges’ operationalization of the policy definitions 
and the factors that influence item difficulty. To assist the judges in generating 
consistently- scaled ratings, the rating process was repeated twice, with feedback. 
Information on consistency among different judges and on the difficulty of each item 2 was 
fed back into the first repetition (round 2), while information on consistency within each 
judge’s set of ratings was fed back into the second repetition (round 3). The third round 
of ratings. permitted the judges to discuss their ratings among themselves to resolve 
problematic ratings. The mean final rating of the judges aggregated across multiple-choice 
(MC) and short constructed-response (SCR) items yielded the threshold values for these 
items in the percent correct metric. These cut scores were then mapped onto the NAEP 
scale (which is defined and scored using item response theory, rather than percent correct). 
For extended constructed-response (ECR) items, judges were asked to select student papers 
which exemplified performance at the cutpoint of each achievement level. Then for each 
achievement level, the mean of the scores assigned to the selected papers was mapped onto 
the NAEP scale in a manner similar to that used for the items scored correct /incorrect. 
The final cut score for each achievement level was a weighted average of the cut score for 
the multiple-choice and short constructed-response items and the cut score for the extended 
constructed-response items, with the weights being proportional to the information 
supplied by the two classes of items. The judges’ ratings, in both metrics, and their 
associated errors of measurement are shown below. 



FIGURE A3 



Outpoints for Achievement Levels at 
Grade 4 



THE NATION’S 
REPORT 
CARD 



1992 



rasp 



Trial State Assessment 



; i y; .... 

Illillli ■ :-r 

Level 


Mean Percent 
Correct - MC & SCR 
(Round 3) 


: ■ Mean Paper 

IlifngMtCft;:!' 


Scale Score* 


K&H9I 


Basic 


38 


2.72 


212 


2.5 


Proficient 


62 


3.14 


243 


2.1 


Advanced 


80 


3.48 


275 


8.8 



* Scale score is derived from a weighted average of the mean percent correct (for multiple-choice and short 
constructed-response items) and the mean paper rating for the extended constructed-response items after both 
were mapped onto the NAEP scale. ** The standard error of the scale score is estimated from the difference 
in mean scale scores for the two equivalent subgroups of judges. 



Item difficulty estimates were based on a preliminary, partial set of responses to the national assessment. 
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For each achievement level, exemplar items were selected that reflected the kinds of tasks 
that examinees at or above the level were likely to be able to perform successfully. Panelists 



to select particular ones as exemplary of each achievement level. The items were 
pre-assigned to each achievement level based on the final round of the judges’ rating data, 
and using the following statistical criteria. For any given level, Basic, Proficient, or 
Advanced: 

• items having an expected p-value ^ .501 and £ .750, at this level , were 
assigned to their level; 

• items meeting the criteria at more than one level were assigned to one level 
taking both the expected p-value and appropriateness of the item for one 
of the levels into account, and 

• items with expected p-values ^ .501 were assigned to levels where a specific 
passage had few or no items at that level. 

During the validation process, items were again reviewed. Those that had been selected 
by the original standard-setting panel were grouped into sets of pre-selected items. All 
remaining items in the released blocks that met the statistical criteria, but that were not 
recommended by the original panel , were grouped into a set identified as additional items 
for review. Exercises that had been recommended for reclassification into another 
achievement level category were presented in their original classification for the purpose of 
this review. 

Panelists worked in grade-level groups to review the possible exemplar items. The task 
was to select a set of items, for each achievement level for their grade, that would best 
communicate to the public the levels of reading ability and the types of skills needed to 
perform in reading at that level. 

After selecting sets of items for their grades, the three grade-level groups met as a whole 
group to review item selection. During this process, cross-grade items that had been 
selected as exemplars by two grades (two such items were selected at grades 8 and 12) were 
assigned to one grade by whole group consensus. In addition, items were evaluated by the 
whole group for overall quality. This process yielded 13 items as recommended exemplars 
for grade 4; 13 items as recommended exemplars for grade 8; and 21 items as recommended 
exemplars for grade 12. 



who had rated specific blocks of released items were asked to review those same items again 
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In Chapter 1, Figure 3 provides the final descriptions of the three achievement levels for 
grade 4. Exemplar items, illustrating what students at each level should be able to perform, 



1992 item pool, apply to the current reading assessment framework and will not change 
from year to year (that is, until the framework changes). However, the sample items 
reflective of the levels will need to be updated each time the assessment is administered. 
Figure 4 in Chapter 1 provides the percentage of students at or above each of the three 
levels and the percentage of students below the Basic level. 



are included in Chapter 1. In principle, the descriptions of the levels, though based on the 
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Anchoring the Achievement Levels 

Scale anchoring is a method for defining performance along a scale. NAEP’S overall 
reading proficiency scale was anchored at the three achievement levels — Basic, Proficient, 
and Advanced — to provide descriptions of what fourth-grade students know and can do 
at each level. 



In February, 1993, ETS applied a modified anchoring procedure to the 1992 reading 
achievement levels. As applied to the achievement levels, the anchoring process was 
designed to determine the sets of questions that students scoring at or above each 
achievement level cutpoint could perform with a high degree of success. A committee of 
reading experts, educators, and others was assembled to review the questions and, using 
their knowledge of reading and student performance, to generalize from the questions to 
descriptions of the types of skills exhibited at each achievement level. 

Specifically, a question was identified as anchoring at an achievement level for a given grade 
if it was answered correctly by at least 65 percent of the students in that grade scoring at 
the cutpoint of that achievement level (and, conversely, by less than 65 percent of the 
students scoring at the cutpoints for any lower achievement level). In order to maximize 
the number of questions offered for consideration, the traditional discrimination criterion, 
that required that the chances of success at the next lower level be at least 30 percentage 
points lower, was not used. The modified anchoring procedure enables the entire set of 
assessment questions to be used in describing student performance. 
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By anchoring the achievement level cutpoints, instead of the entire interval, it is possible 
to determine the types of skills exhibited by all students within an interval. Thus, an item 
anchoring at the Basic level cutpoint will be answered correctly by at least 65 percent of 
minimally basic students and will be answered correctly by at least that percentage of 
students in the basic interval. Since the NAEP results are reported in terms of the 
percentages of students at or above each of the cutpoints, it is important to be able to say 
what all students in the interval are likely to be able to do. In contrast, an anchoring 
procedure based on the interval identifies skills that a typical member of the interval (e.g., 
a typical basic student) likely possesses. While we could infer what a typical student in the 
basic interval can likely do, we would not be able to infer the skills of a minimally basic 
student. 

A description of the entire achievement level interval can be inferred by comparing the 
descriptions for adjacent cutpoints. Thus, the description for the basic cutpoint tells what 
all basic students are likely to be able to do with increasing certainty as their reading 
proficiency increases. The description of the proficient cutpoint refers to the abilities of 
minimally proficient students, but also provides information about the capabilities of basic 
students scoring at the top of the basic interval. To extend the description of the Advanced 
achievement level, since that interval does not have an upper boundary, an additional set 
of questions were identified as almost anchoring at the Advanced level. These questions 
had probabilities of success between 50 and 65 percent for minimally advanced students 
and identify the types of skills that more advanced students are likely to possess. 

The anchoring process was further informed by results using the item mapping procedure. 
Item mapping provides additional information about the performance of students within 
each of the achievement level intervals, and of students who performed below the Basic 
level. In item mapping, the items are arranged in the order of the proficiency level 
corresponding to a defined expected probability of success based on the Item Response 
Theory parameters. The items, or short descriptions, are then displayed, along with the 
proficiency value associated with the selected probability of success. For consistency with 
the anchoring process, a .65 expected probability of success was used. 

To provide a sufficient pool of respondents in identifying anchor items, students at the 
cutpoint of each achievement level were defined as those whose estimated reading 
proficiency was within 12.5 points of the achievement level cutpoint. This is consistent 
with previous anchoring procedures and provides an empirical estimate of the performance 
of students scoring at the cutpoint. To provide stable estimates, the calculations of the 
chances of success on an item had to be based on at least 70 students in the cutpoint 
interval; this is a reduction from the previous requirement of 100 students to accommodate 
the small number of students reaching the Advanced level. 
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Because the extended construeted-response items were scored on an ordered scale with 
4 scoring levels (unsatisfactory, partial, essential, and extensive), the above procedure, 
which relies on the notion of a correct or an incorrect response to an item, was generalized. 
To fit into the anchoring framework, each extended construeted-response item was treated 
as three distinct items corresponding to scores of partial or better, essential or better, and 
extensive. These distinct items were then analyzed in the same manner as items scored 
correct /incorrect. Thus, for example, an extended construeted-response item might anchor 
at the Proficient level for partial or better responses, and at the Advanced level for essential 
or better responses. 

The items were placed in notebooks by grade in the following order: anchored at Basic, 
anchored at Proficient, anchored at Advanced, and almost anchored at Advanced (chance 
of success between 50 and 65 percent at the Advanced level). For cross-referencing 
purposes, the remaining items in the assessment were also included in the notebook under 
the u did not anchor” heading. (These were the items answered correctly by fewer than 
50 percent of the students at the advanced cutpoint.) Each item was accompanied by its 
scoring guide (for construeted-response items), the chance of success on the item for 
students at each achievement level, the counts and weighted proportions of students at each 
level, the overall percent correct on the item for the total population of respondents, and 
the reading purpose and stance classifications for the item. 

Twenty reading education experts participated in the anchoring process. They represented 
teachers for the various grade levels involved, college professors, state curriculum 
supervisors, and researchers. The panelists were divided into three groups, one for each 
grade level. The grade-level groups worked independently for the most part, with periodic 
meetings across the three groups to reconcile views. With the framework for the 1992 
reading assessment and the achievement level descriptions as a reference, panelists were 
asked to use the information in the anchor item notebooks and from the item mapping to 
describe the knowledge, skills, and reasoning demonstrated by the students at the cutpoint 
of each achievement level. In addition, performance as depicted by the maps or items that 
almost anchored was taken as indicating beginning or emerging skills for students in the 
interval. Based on the items anchoring at each level and the item maps, the panelists were 
asked to draft a description of achievement at each level, which is shown in Figure A4. In 
drafting these descriptions, the panelists were instructed to consider the context of the 
assessment and not to overinfer skills from limited numbers of items. 
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The following descriptions of reading behaviors characterize students’ performance on the 
1992 NAEP reading assessment. Based on a thorough review of all the items and how 
students performed at each of the three achievement levels (basic, proficient, and advanced), 
these descriptions were developed to portray the reading ability of students at grades 4, 8, 
and 12 as observed on the NAEP reading assessment. Students’ interactions with the three 
different types of text used in the assessment were considered separately. Each of the three 
text types represents a different purpose for reading. At grade 4, students were asked to read 
for literary experience and to gain information. The descriptions are intended to be 
cumulative from Basic level performance through Advanced. Therefore, demonstrated 
ability at the Proficient level presumes Basic level performance, and Advanced performance 
presumes Proficient, as well as Basic, abilities. 








Grade 4 students understand uncomplicated narratives and high-interest 
informative texts, identify obvious themes, locate explicit information, summarize 
parts of text, and make judgments about characters’ actions. 



Fourth-grade students at the basic level can read uncomplicated narratives with understanding. The literary 
texts at this level include fables and realistic fiction about familiar topics. These students can answer questions 
that focus on specific parts of the story. They are able to identify an obvious theme or message. They can take 
the perspective of characters that are familiar or similar to themselves and compare characters to each other. 
In addition, they can relate to the feelings of familiar characters, as well as interpret and make judgments about 
the characters’ actions. 

Students at the basic level are able to gain information from high-interest informative texts. These students are 
successful when texts are structured as narratives and deal with relatively familiar topics. Students can search 
for and locate explicit information within the text, as well as provide evidence of straightforward comprehension 
of the text. They are able to select relevant information in order to provide a summarization focusing on part 
of the text. They can build simple inferences based on specific information. These students also are able to 
construct their own simple questions related to the passage. 




Grade 4 students understand and Interpret less familiar texts, provide textual 
support for interpretations, generalize across text, identify relevant information, 
understand subtleties in aspects of a story, relate text to background 
experiences, and formulate simple questions. 



Fourth-grade students at the proficient level can form an understanding and extend the meaning of more 
difficult, unfamiliar literary pieces — those in culturally different or historical settings. They are able to respond 
to questions that require some interpretation. Some can construct responses to the story as a whole, as well 
as consider subtleties in aspects of the story. When given interpretations of the story, they can provide some 
justification and support for those interpretations. They are able to recognize multiple perspectives. In 
addition, they have the ability to connect information in the story to the author’s purpose, as well as consider 
alternate possibilities for the story’s development. 

Students at the proficient level are able to gain information and to interpret the meaning of informative text that 
contains narrative elements and direct quotes. Their responses to increasingly more challenging questions 
provide evidence that they can search for, locate, select, prioritize, and apply relevant information. They can 
generalize across parts of the text. They can relate information from the selection to their own background 
experiences and to inferences that are provided for them. They also are able to recognize an author’s basic 
organizational pattern. 
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Grade 4 students interpret and examine the meaning of text, summarize 




information across whole texts, develop their own ideas about textual 




information, understand some literary devices, and are beginning to formulate 


v / ; - ■ 


more complex questions about text. 



Fourth-grade students at the advanced level can form an understanding of what they read and extend, elaborate, 
and examine the meaning of literary texts. They can construct responses to a story by selecting relevant 
information and building their own interpretations that remain consistent with the text. They are able to 
summarize information across the whole story. They understand some literary devices, such as figurative 
language, and can interpret the author’s intentions. 

Students at the advanced level can gain information from what they read and can extend, elaborate, and 
examine the meaning of informative texts about less familiar topics. They are able to read for the purpose of 
gaining a more thorough understanding of a particular topic, and some can develop their own ideas based on 
the information presented in the passage. They can discriminate the relative importance of ideas in the text and 
are beginning to form more complex questions about the selection. They are able to provide an explanation 
of the author’s techniques for presenting information. 
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Data Appendix 

For each of the tables in the main body of the report that presents reading proficiency 
results, this appendix contains corresponding data for each level of the four reporting 
subpopulations -- race/ethnicity, type of community, parents’ education level, and gender. 
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tableau I Teachers’ Reports on Time Spent 
(continued) I Teaching Reading (Per Day) 



45 Minutes or Less 



60 Minutes 



90 Minutes or More 




The NAEP reading scale ranges from 0 to 500. The standard errors of the statistics appear in parentheses. It 
can be said with about 95 percent confidence that, for each population of interest, the value for the entire 
population is within ± 2 standard errors of the estimate for the sample. In comparing two estimates, one must 
use the standard error of the difference (see Appendix A for details). ! Interpret with caution — the nature of 
the sample does not allow accurate determination of the variability of this statistic. *** Sample size is 
insufficient to permit a reliable estimate (fewer than 62 students). 
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:)'42(:3.al »i9|(fel) fe 

228 ( 2.8) 219 ( 2.2) 

30 ( 6.2) 24 ( 4.3) 



190(3.3) 



200,{ 2.9) >; 
26 ( 5.8) 22 ( 4.4) 

fi it§ : *** (**.x 

42 ( 4.9) 16 ( 3.7) 

200 ( 3.5) 208 ( 6.2)1 

20(7.0) 28 ( 7.3) 

flltiSilj : ! 12 ( X’l )■; | 

444 (-.*) ;**•. <“. 4 ) 



ii**- (**;.*:) 

34 ( 5.6) '21 ( 7 . 2 ) . 

183 ( 7.9)! 200 ( 3.8)' 

21 ( 6.5) 35 ( 7.2) 

206 ,(i7i4)l 216(3.2)' 

HI ( 7.8) 7 ' 14 ( 4.4) 

§2©te6;9) 28(3.9) 

209 : (l3S)i.^2’|^|2i)S 

22b-:(: i 2:7)^S216'(---2)5)fc i i ; 
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TABLE A 12 
(continued) 



Teachers’ Reports on Emphasis Given to 
Specific Methods for Teaching Reading 



Phonics 


Integration of Reading and 
Writing 


Whole Language 


Heavy 

Emphasis 


Little or No 
Emphasis 


Heavy 

Emphasis 


Little or No 
Emphasis 


Heavy 

Emphasis 


Little or No 
Emphasis 



TOTAL 


pj^p^gShjd:; 

■ 


: ^^^^Percentage 

^Proficiency 7 - 


State 


'•■15 ( 


31 (43)0) ;*■ ; 


SS3I(E6)IS1 


Nation 


,.2O^|ii3)^®0112iO):teE 


,213 (2)1) 

iH(*2T)..' 


4205;::(2.9)! ., 


PARENTS' 


206 ( 2.9) . 221 

' : '■ ' . ■ % M |^SS £§w |if|| ) 




211 (5.4)1 


EDUCATION 


' 






College graduate 


■1): : ll^ '^)l) ; ) :) ■-;)l)) : ) )' S;:;! . iv )w; 


ll ' '■/"' . 




State 


^lfe(f3.2)SJ#^2>:(;|.4i¥r : l 


32 ( 3.6) 


mmmWSi 


Nation 


210(6.7)! 226(3.0) 


|220 ( 3)2)' • .1 
57 ( 3.2) : T-4 


|%2?(|Q.8)S¥| 


Some after HS 


210 


228 s : (-^)8)/^. 


^ * | 


State 


15 ( 3:4) 1 !8 24 ®Sij8 


29 ( 4.2) 




Nation 


4** * * *-* ,‘W* 4^ ’ 

lltfSlj 43:3.4) 


1223; ( 4.0)% 

&5^KPiS! 




HS graduate 


*“(“.*) 226 ( 5.1) 


225 ( 4.1) : 


4 44- 1 #4 » ^ 


State 


13 ( 3.0) 21( 3.2) . 


. 30 ( 3.7) 


.16 ( 3.5) ; 


Nation 


Bippampp 

8 ( 1.8) 36 (liDlf 


|2l2i(I-3|Jt^: 

i^rr^4)® 


|2Wtyi;9)®Mi 

iW^8i8 


HS non-graduate 


ji^;(iagll2isa|^)» 






State 


17(3.6) 19(4.3) 


30 ( 3.9) ; 


19 ! 4.2! 


Nation 


4* 4 ^4 4 4 J * *'>" ! 4-t* 4^j . 

10 ( 2 5) 34 i 5.0). 


... p* 

48 ( 4.5) 


SBilSIB 


1 don’t know 


4*4 ^44 * . * * *■ / 4 ^ 

• 


202 ( 5.4) 




State 


s.16f( 2.8) v 22 ( 3.0) 


32 ( 3.5) ; ) 


^1^12.5)711 


Nation 


T 98l|ii)lliS20li3l6)ip 
«4i(JlT9) . Jfr 38 H 3.0) • 


, 206 (2:7) - ; 
1-53 ( 2)9) ’’ 


H®(i§)ail 


GENDER 


199 ( 3.1) 215 ( 2.6) 






Male 






State 




. 32 (3.2) ■' 


i 13;.(,2.6) “. . .. 


Nation 


■199 pKg)a^li(i2ii)|fc 


l;209i( 2)4) 
p|§pl)4p 


¥200()3.8 )( ;j? 


Female 


201|(3.3) t! 218 ( 2.8) 


215 ( 2.4) ■' 




State 


22 


l : 3T(3iXv7i' 




Nation 


208i(-4:7;): 219j{i2;8) 

1?X~.1 )4)’. 1 '40; ( *2.3)>% 


217 ( 2.6) : 


S2O0J^5)i* 




l2Hl'(y<9))S^ ! 2S5;:(' 2.7f.t 


i225' ; (l2l3)iil 


■> : 216)(45)1;)r 



V 

: r. ■ ?v : - ;.< ' 



mmmi 
sllci® | 

219 ( 2.6) 



WSM§ 3)3)11 

ai5 ( i.9) 

215 ( 2.0) 



;21'3a(#S)1 

spipii)§ 

16 (®®| 

RKI 

227 { 5.0) 

' 19.1^8 
2dlfpS)l 

&$$08ik 

212(1^0- 

20 ( 4.7) 

37 ( 5.7) 

1%t;|rf|^Sp 

23 ( 3.8) 

202J(#6)S 

^i-3.2)t- 

MM02.5):, 



( ; 22l(li!6)4 

SoiiP;5)5l 

42 (3.3) I 



8j 

m 



:!221 ( 2.6 

iM i5T,ii7)^ 

221 ( 3.4) 

30 ( 4.1) : 
226 ( 4.6) '' 
4|si9"( 3.4) S 

s 

Sri 30.( 40) : 

: 214 ( 3.3) ' 

216'( 4.9) | 



- -'■- 34 ,{ 

1 i g® 

* *-*- ^ 

1 27 ( 

p206tj 

l^aoif 

^240| 

IS IS 

Sr) p 

I 2s|( 

- 210 ( 

Slli® 

212:'( 



s:i) 

6.3).., 

P6) . 

w,?&< 

3,2} 1 

mm 

ii3)|p 

ill fi 

mym 

Utii % 

w$m 



S?^£3:2)^^</36|( 3.6) 

* v5 ‘ 






The NAEP reading scale ranges from 0 to 500. The standard errors of the statistics appear in parentheses. It 
can be said with about 95 percent confidence that, for each population of interest, the value for the entire 
population is within ± 2 standard errors of the estimate for the sample. In comparing two estimates, one must 
use the standard error of the difference (see Appendix A for details). The percentages may not total 
100 percent because the “Moderate Emphasis” category is not included. ! Interpret with caution - the nature 
of the sample does not allow accurate determination of the variability of this statistic. *** Sample size is 
insufficient to permit a reliable estimate (fewer than 62 students). 
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table Ai 2 I Teachers’ Reports on Emphasis Given to 
(continued) I Specific Methods for Teaching Reading 



Literature-based Reading 


Reading Across the 
Content Areas 


Individualized Reading 
Programs 


Heavy 

Emphasis 


Little or No 
Emphasis 


Heavy 

Emphasis 


Little or No 
Emphasis 


Heavy 

Emphasis 


Little or No 
Emphasis 



TOTAL 

State 

Nation 



RACE/ 

ETHNICITY 



White 

State 

Nation 

Black 

State 

Nation 

Hispanic 

State 

Nation 

Amer. Indian 

State 

Nation 



TYPE OF 
COMMUNITY 



Disadv. urban 

State 

Nation 

Extreme rural 

State 

Nation 

Other 

State 

Nation 



§§j ^rofici«icy 



• A" 20' (-2lS) ' 

SBso 

220 (-2.0) A =208 ( 3.2) 



23 ( 2.3) 

I!2'2l|p2')2)fa 

^52ti v £9)?fj 

228 ( 2.1) 

>«! S =' S 

fe®5;2) ! g ; 

190J( 317)1:;) 

flli^.'DY: 

193 ( 2.8) 

35(5.1) 

47 (3>) 
<198(3.4) 

I'll 

*«* I"** * j 

■til 6 . 6 ) 

* •»•* l * * k ) ■ 

a. 

iSS <*m ■ 

' 26 ( 5.1) 

IsSStfaSSils 

||2||jg$)3| 

25 ( 3.1) 

21^314)4 

SsifMip 

•220 ( 2.3) 



11 ( 2.1) 
214 ( 3.6) 

18 ( 3.8 ) r 

191 ( 4.1)1 : 
16 ( 3.0) 

192 ( 3.1)1 

A 15 ( 3.4) 

s ***» 1*4 '. 

11 ( 2 . 6 ) 
206 { 5,1)1 

20 ( 6 . 1 ) 

wMm 



■.;;-212 ( 4rl )! .Y . 

. - 2C«(|f'.9)!|| 
19 ( 3.2) 

Stilt): ( 2.0) 

g|f*209 i (;'0:5,);f$? 



^^SjY.Pirowtaae'/ :AY A 

|®^||RroRci«icy : V. ■ ■ 

, p :• --a, /A--. : j | ^ : • \ a'A.A.a.- 

' ; ir (-ISi'rl 

49 

.216 ( 2.0) 214l(i4.4)!'.^ 



.37 ( 4.0) 
■222C0-9) 

■'49:0 3>1 )> 

224 ( 2.3) 

43 ( 5.9) 
190(2.9) 

m&em 

191 (2.2) 

: : ; : 

41 ( 6.4) 
190(:5.8) 

:i36.((-3.4)'"- : 

198 ( 3.6) 

44 i 7.6) 

45 ( 7.4) 

■4HH ^ 



60 (16.3)! 
198 (-7.3)! M 

1 86 (j®|k||| 

214 ( 5.3)! | 

t08§g$H v '4S 

liiiSai 

^19H(|2i2)f»| 



17 ( 3.2) 
218 ( 2.6)! 

l§f§l(f2&)!§l 

jf||; ( 4.5)1 

16 ( 4.2) S : 
194 ( 4.3)1 
?,:(2.3) . 
•*+* |*» 

13(3.5) 

+** ( 4 *^ ) 

12 ( 3.3) 

14(4.4) 

+*» (****■) 

14 ( 4.2) :: 

*** I* » 



St 6 ( 6.4)*“* 

**-* ^4 + 

ilS(§&)7 

* * » * 4 j 

20 ( 6 . 1 ) 

W&(0rf'$ 

17(3.6) 

^KiiK 



afcag Percentage ; < 



pS5^A)^1 

216 ( 3.5) ' 



aisiiil 
■ .a" n ■ : a a 



11 ( 1.6) 

iiSIitt 

iiia»4i-.9)f-- 

15 ( 3:8) < 
184 ( 5.6)! 

12 ( 2 . 8 ) 
196 ( 4.3)1 

13 ( 2.5) 

lr*>* £+* 

~i7(*3.0K 
192 ( 5.4)! 

8 ( 2.3) 

♦+1 £•*♦* *y 

| ;$ ( 3 : 2 y : 

*4* 



:,J5 V (.8:6)! 

v : *** 

187 ( 6.4)! 

|205tl®O) 

; 220?(y3,9)|S 



^ 49 ( 3.0)<^ 

gg|yi.8r ; 

50 ( 3.5) . 

I &o!0&)im 

: m#&w 6 

225 ( 2.1) 

f ; '“42 \ : sjy [ -r 

196 ( 3.3) 

197 { 2.7) 

- 45(- : 5,3) Si 

rti®pS)li| 

206(3.6) 

"57 $5)11 

■ *r*A i** 

' 67 (. 7.6) 

+ |4I j 



#^1(16.3)1 

»*•*• l*+ * ^ 

^^(?65)' . 

: 19^^ 

52 (7.0) 

^'^f«A)i 
.55( 9!8)ifj 

memsm . | 

49 ( 3.9) 

. 214 ( L9LS: 

* M 

I«21.9I(i2lipa 
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Teachers’ Reports on Emphasis Given to 
Specific Methods for Teaching Reading 



1992 

Trial State Assessment 



Literature-based Reading 


Reading Across the 
Content Areas 


Individualized Reading 
Programs 


Heavy 

Emphasis 


Little or No 
Emphasis 


Heavy 

Emphasis 


Little or No 
Emphasis 


Heavy 

Emphasis 


Little or No 
Emphasis 



TOTAL 



State 

Nation 



PARENTS' 

EDUCATION 



College graduate 

State 

Nation 

Some after HS 

State 

Nation 

HS graduate 

State 

Nation 

HS non-graduate 

State 

Nation 

I don’t know 

State 

Nation 



GENDER 



Male 

State 

Nation 

Female 

State 

Nation 



PeroelStage 

Proficiency 



212*2:6) 

'220X2.0) 



Wi0t~2£)U 

•208T3.2) : 

77X777, 77;; : ;;7g : . 



26 (3:1)1 
:218)(3.3) 

54 ( 3 S)i 
228 ( 2.8) 

20 ! 3.3) 

4 * * {* * * ) 

55 ( 4.9) 
225 ( 3.6) 

22 ( 2 . 6 ) 
:269l(|3l8) 
48 ( 4.7) 
214 ( 2.9) 



18 I 
216 (4.4) ; 

216(5.1)! 

V 25 ( 4.2) 

v. iMpM! 

19(3.0) 

^testeerp4.:9) 

■»S i r 

— f-.M 

22 ( 47) 

** * { *•*'/ } • 

. 12 ( 2.5) 



1 






Pilf&Ii 

iplllil 

195 ( 6,2) 

. 7;x 

■26 (i;i9Bllp(p3:i)^ 

206 ( 2.9) ^2^13:1)^ 

21 ( 2.9) 

^®^6l0l6)l# 

7:8"4--- v 8 t'4c 

■ ■ 20 ( 3.0) : . 
*§®p!2)* 
12 ( 216).'. 
214(3.5) 



209 ( 2)9) 

isjwiS) 

S^6)' 

il5.#Tti) 



PlPeiwiHageg) 

iSproficieshcy-^ 



2i2.(:;2.2) 

IP|(I2- ; 74| 



214 ( 4.4)1 ; 

18 ! 3.3 i 1 
jj 226 ( 6.9)! 7; 



1546^^)1 

217 ( 3.1) 

50 ( 3:2)| 

224 ( 3.0) 

W * 

)222)(:3.8L- ' 

'■5T(J$8)i 

226 ( 3.6). 

33 ( 4.1) 

“^8tf§§|| 

.21.1 (2.9). 



41 ( 4.5) 

ms$m 

195 ( 4.3) 

|3?|S# 

I205 ; :(l2l6);i 

2i °i$S®s 



:;«.:39-(;3.8)| 

ism&ir. 

i®)(f3';0)"7 

|2§fjp$g 

wmmm 

■:2l5l(s2.3)l 
221 ( 2.2)1 



t§?f^(f2.6)?pi 

. •*•(*’■•) 

17(3.7) 

'^212i(..3.^)!li|: 

■I ppiS SV I r 

■■ *+•*-../*•*• * \ ■ 

(Ol 

m -’13 ((Ml ! 

... (...) 

18S#6l(!2s7) •$ 

• H- |H Jj 

. 16 ( 3.0) 

W205 ( 4.9) | ' 

smm 2:4) 

;^>QB*4.8)! ■; 



S®209( : 4.'1). 

8 9 ( 2.3ff« 

?fe:i8;(f3.iT' 

214(3.7) 

220 ( 5.1)! 



(Proficiency o ;; 

/: .; 7. -8 ;8'88:8’ : ) 

?205'T4.4) ^21 3 K§&:)f® 

1PIS#:6) 

216 ( 3.5) 219 (;1.8);. «■ 

x ■7 ;: 8 :1v 






:x:! 



W 



Mmm % (iSliaiiB 

1 10 (2.0) 

^53|(f9T2)S 



. .. 49®>'.4) ; ?S 



iwtp^si 
HK$S» 
2 8 f &m 

*-*|r^*-*J 

12 ( 1.8) 
*** (■*.*) 

.10 ( 2.6);,' 



- 10«|2SS 

MtM*)' 

?3199((^!0)4' 

't:i3](>'2‘:0) ■. 

210 ( 4.2) 



Hpfi 

W22«I(|2.#® 

: .■ r | 

l|Sl^ : 4.9)' 

^^|3:4)fc3| 

: v 53 ( 4.2). I 

p2i:ii#2:5pi 

54 ( 3.8) 

215 ( 3.8) ;. 

' ; 48 ( 4.5)7 ■ 

#2(S;(!^1M 

1 49 ( 4.9) 

' 1 199 (56(8)%** 

Ilp^l2!0» 









v 7 :: ) 77; ;p 



||r::7; : S)7;:777 

1? ( 1-5) . 
,200i( : '4.5){ • 

Pbfpf6).8 

UWw^) i 

Sii,Tii) 

ii'Sipts)^ 

22i(:(i3.6) 7 



7 7)7 






, ..48)^l|fe 
(8210 plfEJ 
^55|p3:6)!«I 

^2154(^.9) T.;j 

49(3.3) 

IStMilM 

^53 ; (|2i9)v a 

223 ( 2,i||‘ , “‘ 



The NAEP reading scale ranges from 0 to 500, The standard errors of the statistics appear in parentheses. It 
can be said with about 95 percent confidence that, for each population of interest, the value for the entire 
population is within ± 2 standard errors of the estimate for the sample. In comparing two estimates, one must 
use the standard error of the difference (see Appendix A for details). The percentages may not total 
100 percent because the “Moderate Emphasis” category is not included. ! Interpret with caution - the nature 
of the sample does not allow accurate determination of the variability of this statistic. *** Sample size is 
insufficient to permit a reliable estimate (fewer than 62 students). 
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table M3 | Teachers’ Reports on Instructional 
I Materials for Reading 



Primarily Basal 



Primarily Trade 
Books 



Both Basal and 
Trade Books 



Other 
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^PwsiSfitage, 1|1 
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S 
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1! 
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Nation 
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